generated from coulomb/repo-seed
Add report fragments and export manifest
This commit is contained in:
@@ -42,9 +42,10 @@ The same CLI contracts are packaged by the container baseline. See
|
|||||||
[docs/CONTAINER.md](docs/CONTAINER.md). The dependency-light local API wraps
|
[docs/CONTAINER.md](docs/CONTAINER.md). The dependency-light local API wraps
|
||||||
those contracts for service and container operation; see
|
those contracts for service and container operation; see
|
||||||
[docs/LOCAL-SERVICE-API.md](docs/LOCAL-SERVICE-API.md).
|
[docs/LOCAL-SERVICE-API.md](docs/LOCAL-SERVICE-API.md).
|
||||||
New runs write a richer `sources.lock.json` and
|
New runs write a richer `sources.lock.json`,
|
||||||
`reports/submission-package.json` alongside the assessment package so reviewers
|
`reports/submission-package.json`, extension report fragments, and
|
||||||
can inspect source, metadata, artifact, and boundary references.
|
`exports/export-manifest.json` alongside the assessment package so reviewers can
|
||||||
|
inspect source, metadata, artifact, export, and boundary references.
|
||||||
|
|
||||||
The `sample-noop` extension exercises the guide-board contracts without invoking
|
The `sample-noop` extension exercises the guide-board contracts without invoking
|
||||||
an external harness. `sdk-fixture` demonstrates the extension SDK contracts for
|
an external harness. `sdk-fixture` demonstrates the extension SDK contracts for
|
||||||
|
|||||||
@@ -653,6 +653,18 @@ building complex runtime code.
|
|||||||
- `reported_metadata`
|
- `reported_metadata`
|
||||||
- `certification_boundary`
|
- `certification_boundary`
|
||||||
|
|
||||||
|
### `ExportManifest`
|
||||||
|
|
||||||
|
- `export_type`
|
||||||
|
- `source_package_ref`
|
||||||
|
- `source_lock_ref`
|
||||||
|
- `summary`
|
||||||
|
- `policy_summary`
|
||||||
|
- `mapping_summary`
|
||||||
|
- `report_fragments`
|
||||||
|
- `counts`
|
||||||
|
- `certification_boundary`
|
||||||
|
|
||||||
## Result Vocabulary
|
## Result Vocabulary
|
||||||
|
|
||||||
The evidence model should allow these statuses:
|
The evidence model should allow these statuses:
|
||||||
@@ -740,9 +752,11 @@ runs/<run-id>/
|
|||||||
mappings.json
|
mappings.json
|
||||||
reports/
|
reports/
|
||||||
report.md
|
report.md
|
||||||
|
fragments.json
|
||||||
assessment-package.json
|
assessment-package.json
|
||||||
submission-package.json
|
submission-package.json
|
||||||
exports/
|
exports/
|
||||||
|
export-manifest.json
|
||||||
```
|
```
|
||||||
|
|
||||||
## Container And Service Model
|
## Container And Service Model
|
||||||
@@ -836,6 +850,8 @@ hooks for runners and normalizers.
|
|||||||
6. Add container design after the CLI baseline is stable.
|
6. Add container design after the CLI baseline is stable.
|
||||||
7. Add optional service API around the CLI job model.
|
7. Add optional service API around the CLI job model.
|
||||||
8. Add OSCAL export and procedural evidence-pack support after the internal
|
8. Add OSCAL export and procedural evidence-pack support after the internal
|
||||||
evidence model proves itself with executable extensions.
|
evidence model proves itself with executable extensions. The first generic
|
||||||
|
export is `exports/export-manifest.json`; authority-specific interchange
|
||||||
|
remains extension-owned until the internal model is stable.
|
||||||
|
|
||||||
The first extension SDK contract is documented in `docs/EXTENSION-SDK.md`.
|
The first extension SDK contract is documented in `docs/EXTENSION-SDK.md`.
|
||||||
|
|||||||
@@ -79,6 +79,7 @@ A completed CLI command prints a JSON result with:
|
|||||||
- `assessment_package`: JSON assessment package path,
|
- `assessment_package`: JSON assessment package path,
|
||||||
- `report`: Markdown report path,
|
- `report`: Markdown report path,
|
||||||
- `submission_package`: portable submission package manifest path,
|
- `submission_package`: portable submission package manifest path,
|
||||||
|
- `export_manifest`: portable generic JSON export manifest path,
|
||||||
- `retention_summary`: compact durable summary path.
|
- `retention_summary`: compact durable summary path.
|
||||||
|
|
||||||
The output directory uses this contract:
|
The output directory uses this contract:
|
||||||
@@ -95,7 +96,9 @@ normalized/findings.json
|
|||||||
normalized/mappings.json
|
normalized/mappings.json
|
||||||
reports/assessment-package.json
|
reports/assessment-package.json
|
||||||
reports/report.md
|
reports/report.md
|
||||||
|
reports/fragments.json
|
||||||
reports/submission-package.json
|
reports/submission-package.json
|
||||||
|
exports/export-manifest.json
|
||||||
artifacts/
|
artifacts/
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -107,6 +110,12 @@ raw artifact manifest, and repeats the certification boundary. It is a portable
|
|||||||
handoff manifest for preparation evidence, not an authority-specific final
|
handoff manifest for preparation evidence, not an authority-specific final
|
||||||
submission.
|
submission.
|
||||||
|
|
||||||
|
Extension report fragments are recorded in `reports/fragments.json`, embedded in
|
||||||
|
`reports/assessment-package.json`, and rendered into the Markdown report.
|
||||||
|
`exports/export-manifest.json` is the first generic portable export surface. It
|
||||||
|
is derived from the assessment package and carries summary, policy, mapping,
|
||||||
|
fragment, count, source-lock, and boundary references.
|
||||||
|
|
||||||
Use the retained run helpers for history:
|
Use the retained run helpers for history:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
@@ -121,6 +130,10 @@ PYTHONPATH=src python3 -m guide_board runs trend --runs-dir runs
|
|||||||
PYTHONPATH=src python3 -m guide_board runs gate --runs-dir runs
|
PYTHONPATH=src python3 -m guide_board runs gate --runs-dir runs
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Trend summaries include status changes, unexpected finding deltas, unresolved
|
||||||
|
review deltas, mapping target deltas, evidence result deltas, and a compact
|
||||||
|
human-readable summary string for each target/assessment pair.
|
||||||
|
|
||||||
## Local Service Flow
|
## Local Service Flow
|
||||||
|
|
||||||
Start the service from the guide-board repository:
|
Start the service from the guide-board repository:
|
||||||
|
|||||||
@@ -62,7 +62,8 @@ The script:
|
|||||||
- builds `guide-board-core:smoke`,
|
- builds `guide-board-core:smoke`,
|
||||||
- mounts a host output directory at `/runs`,
|
- mounts a host output directory at `/runs`,
|
||||||
- runs the bundled sample assessment,
|
- runs the bundled sample assessment,
|
||||||
- verifies that the expected run artifacts are present on the host.
|
- verifies that the expected run artifacts, report fragments, submission
|
||||||
|
manifest, and generic export manifest are present on the host.
|
||||||
|
|
||||||
Override the runtime, image name, or output directory when needed:
|
Override the runtime, image name, or output directory when needed:
|
||||||
|
|
||||||
|
|||||||
@@ -83,6 +83,8 @@ The key runtime fields are:
|
|||||||
- `normalizers`: optional plug-ins that convert native runner output into the
|
- `normalizers`: optional plug-ins that convert native runner output into the
|
||||||
stable runner-result shape before evidence is written.
|
stable runner-result shape before evidence is written.
|
||||||
- `mappings`: mapping set IDs under `mappings/<mapping-id>.json`.
|
- `mappings`: mapping set IDs under `mappings/<mapping-id>.json`.
|
||||||
|
- `report_fragments`: optional Markdown file or Python module descriptors for
|
||||||
|
extension-owned report content.
|
||||||
- `certification_boundary`: explicit statement of what the extension does not
|
- `certification_boundary`: explicit statement of what the extension does not
|
||||||
certify.
|
certify.
|
||||||
|
|
||||||
@@ -209,6 +211,53 @@ to extension-owned mappings and writes normalized mapping records to:
|
|||||||
runs/<run-id>/normalized/mappings.json
|
runs/<run-id>/normalized/mappings.json
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Report Fragments
|
||||||
|
|
||||||
|
Extensions can contribute report fragments through `report_fragments`.
|
||||||
|
|
||||||
|
Static Markdown file:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "overview",
|
||||||
|
"kind": "markdown_file",
|
||||||
|
"path": "reports/overview.md",
|
||||||
|
"title": "Overview"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Dynamic Python fragment:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "sdk-fixture-summary",
|
||||||
|
"kind": "python_module",
|
||||||
|
"module_path": "reports/sdk_fixture_summary.py",
|
||||||
|
"callable": "build_fragment",
|
||||||
|
"path": null,
|
||||||
|
"title": "SDK Fixture Summary"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Fragment paths are resolved relative to the extension root and must stay inside
|
||||||
|
that root. A Python fragment receives `root`, `run_dir`, `run_id`, `plan`,
|
||||||
|
`evidence`, `findings`, `mappings`, `assessment_package`, `policy_summary`,
|
||||||
|
`source_lock`, `extension_path`, and `report_fragment`.
|
||||||
|
|
||||||
|
It returns:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def build_fragment(context: dict) -> dict:
|
||||||
|
return {
|
||||||
|
"markdown": "### Extension Summary\n\n- evidence items: 2",
|
||||||
|
"structured": {"evidence_count": 2},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Fragments are written to `reports/fragments.json`, embedded in the assessment
|
||||||
|
package, rendered in `reports/report.md`, and summarized in
|
||||||
|
`exports/export-manifest.json`.
|
||||||
|
|
||||||
## Evidence Request Sets
|
## Evidence Request Sets
|
||||||
|
|
||||||
Procedural and hybrid compliance extensions may include evidence request sets
|
Procedural and hybrid compliance extensions may include evidence request sets
|
||||||
@@ -402,9 +451,9 @@ profiles.
|
|||||||
|
|
||||||
## Source Lock And Submission Package
|
## Source Lock And Submission Package
|
||||||
|
|
||||||
Every new run writes `sources.lock.json` and
|
Every new run writes `sources.lock.json`, `reports/submission-package.json`,
|
||||||
`reports/submission-package.json`. Extension authors should treat source
|
and the generic portable export manifest at `exports/export-manifest.json`.
|
||||||
metadata as part of the evidence contract:
|
Extension authors should treat source metadata as part of the evidence contract:
|
||||||
|
|
||||||
- declare extension, authority, framework, runner, and normalizer metadata in
|
- declare extension, authority, framework, runner, and normalizer metadata in
|
||||||
`extension.json` when it is static;
|
`extension.json` when it is static;
|
||||||
|
|||||||
@@ -98,8 +98,8 @@ errors.
|
|||||||
### `GET /runs/{job_id}/reports`
|
### `GET /runs/{job_id}/reports`
|
||||||
|
|
||||||
Returns the Markdown report content, assessment package JSON, retention summary,
|
Returns the Markdown report content, assessment package JSON, retention summary,
|
||||||
submission package JSON when present, and their filesystem paths after a job has
|
submission package JSON, export manifest JSON when present, and their filesystem
|
||||||
succeeded.
|
paths after a job has succeeded.
|
||||||
|
|
||||||
### `GET /retained-runs`
|
### `GET /retained-runs`
|
||||||
|
|
||||||
|
|||||||
@@ -19,6 +19,7 @@
|
|||||||
"waivers",
|
"waivers",
|
||||||
"challenges",
|
"challenges",
|
||||||
"exclusions",
|
"exclusions",
|
||||||
|
"report_fragments",
|
||||||
"certification_boundary",
|
"certification_boundary",
|
||||||
"created_at"
|
"created_at"
|
||||||
],
|
],
|
||||||
@@ -38,6 +39,7 @@
|
|||||||
"waivers": { "type": "array", "items": { "type": "object" } },
|
"waivers": { "type": "array", "items": { "type": "object" } },
|
||||||
"challenges": { "type": "array", "items": { "type": "object" } },
|
"challenges": { "type": "array", "items": { "type": "object" } },
|
||||||
"exclusions": { "type": "array", "items": { "type": "object" } },
|
"exclusions": { "type": "array", "items": { "type": "object" } },
|
||||||
|
"report_fragments": { "type": "array", "items": { "type": "object" } },
|
||||||
"certification_boundary": { "type": "string" },
|
"certification_boundary": { "type": "string" },
|
||||||
"created_at": { "type": "string" }
|
"created_at": { "type": "string" }
|
||||||
}
|
}
|
||||||
|
|||||||
36
docs/schemas/export-manifest.schema.json
Normal file
36
docs/schemas/export-manifest.schema.json
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||||
|
"title": "Guide Board Export Manifest",
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": [
|
||||||
|
"id",
|
||||||
|
"schema_version",
|
||||||
|
"export_type",
|
||||||
|
"run_id",
|
||||||
|
"created_at",
|
||||||
|
"source_package_ref",
|
||||||
|
"source_lock_ref",
|
||||||
|
"summary",
|
||||||
|
"policy_summary",
|
||||||
|
"mapping_summary",
|
||||||
|
"report_fragments",
|
||||||
|
"counts",
|
||||||
|
"certification_boundary"
|
||||||
|
],
|
||||||
|
"properties": {
|
||||||
|
"id": { "type": "string" },
|
||||||
|
"schema_version": { "type": "string" },
|
||||||
|
"export_type": { "type": "string" },
|
||||||
|
"run_id": { "type": "string" },
|
||||||
|
"created_at": { "type": "string" },
|
||||||
|
"source_package_ref": { "type": "string" },
|
||||||
|
"source_lock_ref": { "type": "string" },
|
||||||
|
"summary": { "type": "object" },
|
||||||
|
"policy_summary": { "type": "object" },
|
||||||
|
"mapping_summary": { "type": "object" },
|
||||||
|
"report_fragments": { "type": "array", "items": { "type": "object" } },
|
||||||
|
"counts": { "type": "object" },
|
||||||
|
"certification_boundary": { "type": "string" }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -142,7 +142,23 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"mappings": { "type": "array", "items": { "type": "string" } },
|
"mappings": { "type": "array", "items": { "type": "string" } },
|
||||||
"report_fragments": { "type": "array", "items": { "type": "string" } },
|
"report_fragments": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": ["string", "object"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["id", "kind"],
|
||||||
|
"properties": {
|
||||||
|
"id": { "type": "string" },
|
||||||
|
"kind": { "type": "string", "enum": ["markdown_file", "python_module"] },
|
||||||
|
"path": { "type": ["string", "null"] },
|
||||||
|
"module_path": { "type": ["string", "null"] },
|
||||||
|
"callable": { "type": ["string", "null"] },
|
||||||
|
"title": { "type": ["string", "null"] },
|
||||||
|
"description": { "type": ["string", "null"] }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"dependencies": { "type": "array", "items": { "type": "string" } },
|
"dependencies": { "type": "array", "items": { "type": "string" } },
|
||||||
"restricted_assets": { "type": "array", "items": { "type": "string" } },
|
"restricted_assets": { "type": "array", "items": { "type": "string" } },
|
||||||
"certification_boundary": { "type": "string" }
|
"certification_boundary": { "type": "string" }
|
||||||
|
|||||||
@@ -21,6 +21,7 @@
|
|||||||
"created_at": { "type": "string" },
|
"created_at": { "type": "string" },
|
||||||
"summary": { "type": "object" },
|
"summary": { "type": "object" },
|
||||||
"report_refs": { "type": "array", "items": { "type": "string" } },
|
"report_refs": { "type": "array", "items": { "type": "string" } },
|
||||||
|
"export_refs": { "type": "array", "items": { "type": "string" } },
|
||||||
"artifact_retention": { "type": "object" }
|
"artifact_retention": { "type": "object" }
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -74,7 +74,17 @@
|
|||||||
"mappings": [
|
"mappings": [
|
||||||
"sdk-fixture-map"
|
"sdk-fixture-map"
|
||||||
],
|
],
|
||||||
"report_fragments": [],
|
"report_fragments": [
|
||||||
|
{
|
||||||
|
"id": "sdk-fixture-summary",
|
||||||
|
"kind": "python_module",
|
||||||
|
"module_path": "reports/sdk_fixture_summary.py",
|
||||||
|
"callable": "build_fragment",
|
||||||
|
"path": null,
|
||||||
|
"title": "SDK Fixture Summary",
|
||||||
|
"description": "Summarizes SDK fixture evidence for report fragment tests."
|
||||||
|
}
|
||||||
|
],
|
||||||
"dependencies": [],
|
"dependencies": [],
|
||||||
"restricted_assets": [],
|
"restricted_assets": [],
|
||||||
"certification_boundary": "SDK fixture only. It does not certify any product, process, or repository."
|
"certification_boundary": "SDK fixture only. It does not certify any product, process, or repository."
|
||||||
|
|||||||
26
extensions/sdk-fixture/reports/sdk_fixture_summary.py
Normal file
26
extensions/sdk-fixture/reports/sdk_fixture_summary.py
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
"""Report fragment for the SDK fixture extension."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
def build_fragment(context: dict) -> dict:
|
||||||
|
evidence_count = len(context["evidence"])
|
||||||
|
finding_count = len(context["findings"])
|
||||||
|
source_lock = context["source_lock"]
|
||||||
|
markdown = "\n".join(
|
||||||
|
[
|
||||||
|
"### SDK Fixture Summary",
|
||||||
|
"",
|
||||||
|
f"- evidence items: {evidence_count}",
|
||||||
|
f"- findings: {finding_count}",
|
||||||
|
f"- source lock: {source_lock.get('id')}",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"markdown": markdown,
|
||||||
|
"structured": {
|
||||||
|
"evidence_count": evidence_count,
|
||||||
|
"finding_count": finding_count,
|
||||||
|
"source_lock_ref": source_lock.get("id"),
|
||||||
|
},
|
||||||
|
}
|
||||||
@@ -42,7 +42,9 @@ for path in \
|
|||||||
"$RUNS_DIR/sample-noop/normalized/mappings.json" \
|
"$RUNS_DIR/sample-noop/normalized/mappings.json" \
|
||||||
"$RUNS_DIR/sample-noop/reports/assessment-package.json" \
|
"$RUNS_DIR/sample-noop/reports/assessment-package.json" \
|
||||||
"$RUNS_DIR/sample-noop/reports/report.md" \
|
"$RUNS_DIR/sample-noop/reports/report.md" \
|
||||||
"$RUNS_DIR/sample-noop/reports/submission-package.json"
|
"$RUNS_DIR/sample-noop/reports/fragments.json" \
|
||||||
|
"$RUNS_DIR/sample-noop/reports/submission-package.json" \
|
||||||
|
"$RUNS_DIR/sample-noop/exports/export-manifest.json"
|
||||||
do
|
do
|
||||||
if [ ! -f "$path" ]; then
|
if [ ! -f "$path" ]; then
|
||||||
echo "ERROR: expected artifact missing: $path" >&2
|
echo "ERROR: expected artifact missing: $path" >&2
|
||||||
|
|||||||
@@ -8,11 +8,13 @@ from pathlib import Path
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from guide_board.artifacts import build_artifact_manifest, build_submission_manifest
|
from guide_board.artifacts import build_artifact_manifest, build_submission_manifest
|
||||||
|
from guide_board.exports import build_export_manifest
|
||||||
from guide_board.io import load_json, write_json
|
from guide_board.io import load_json, write_json
|
||||||
from guide_board.mapping import build_mapping_records, summarize_mappings
|
from guide_board.mapping import build_mapping_records, summarize_mappings
|
||||||
from guide_board.normalizers import normalize_step_result
|
from guide_board.normalizers import normalize_step_result
|
||||||
from guide_board.planning import build_run_plan
|
from guide_board.planning import build_run_plan
|
||||||
from guide_board.policy import apply_policy
|
from guide_board.policy import apply_policy
|
||||||
|
from guide_board.reports import build_report_fragments, markdown_for_fragments
|
||||||
from guide_board.retention import build_retention_summary
|
from guide_board.retention import build_retention_summary
|
||||||
from guide_board.runners import run_step
|
from guide_board.runners import run_step
|
||||||
from guide_board.schema import assert_valid
|
from guide_board.schema import assert_valid
|
||||||
@@ -64,7 +66,19 @@ def run_assessment(
|
|||||||
applied_exclusions,
|
applied_exclusions,
|
||||||
created_at,
|
created_at,
|
||||||
)
|
)
|
||||||
|
report_fragments = build_report_fragments(
|
||||||
|
root,
|
||||||
|
run_dir,
|
||||||
|
run_id,
|
||||||
|
plan,
|
||||||
|
evidence,
|
||||||
|
findings,
|
||||||
|
mapping_records,
|
||||||
|
assessment_package,
|
||||||
|
)
|
||||||
|
assessment_package["report_fragments"] = report_fragments
|
||||||
assert_valid(assessment_package, "assessment-package")
|
assert_valid(assessment_package, "assessment-package")
|
||||||
|
export_manifest = build_export_manifest(assessment_package)
|
||||||
|
|
||||||
run_metadata = {
|
run_metadata = {
|
||||||
"id": run_id,
|
"id": run_id,
|
||||||
@@ -85,6 +99,7 @@ def run_assessment(
|
|||||||
findings,
|
findings,
|
||||||
mapping_records,
|
mapping_records,
|
||||||
assessment_package,
|
assessment_package,
|
||||||
|
export_manifest,
|
||||||
retention_summary,
|
retention_summary,
|
||||||
)
|
)
|
||||||
return {
|
return {
|
||||||
@@ -94,6 +109,7 @@ def run_assessment(
|
|||||||
"assessment_package": str(run_dir / "reports" / "assessment-package.json"),
|
"assessment_package": str(run_dir / "reports" / "assessment-package.json"),
|
||||||
"report": str(run_dir / "reports" / "report.md"),
|
"report": str(run_dir / "reports" / "report.md"),
|
||||||
"submission_package": str(run_dir / "reports" / "submission-package.json"),
|
"submission_package": str(run_dir / "reports" / "submission-package.json"),
|
||||||
|
"export_manifest": str(run_dir / "exports" / "export-manifest.json"),
|
||||||
"retention_summary": str(run_dir / "retention-summary.json"),
|
"retention_summary": str(run_dir / "retention-summary.json"),
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -419,6 +435,7 @@ def _assessment_package(
|
|||||||
"waivers": applied_waivers,
|
"waivers": applied_waivers,
|
||||||
"challenges": applied_challenges,
|
"challenges": applied_challenges,
|
||||||
"exclusions": applied_exclusions,
|
"exclusions": applied_exclusions,
|
||||||
|
"report_fragments": [],
|
||||||
"certification_boundary": "Guide Board produces preparation evidence only and does not issue certifications or audit assurance.",
|
"certification_boundary": "Guide Board produces preparation evidence only and does not issue certifications or audit assurance.",
|
||||||
"created_at": created_at,
|
"created_at": created_at,
|
||||||
}
|
}
|
||||||
@@ -432,6 +449,7 @@ def _write_run_directory(
|
|||||||
findings: list[dict[str, Any]],
|
findings: list[dict[str, Any]],
|
||||||
mapping_records: list[dict[str, Any]],
|
mapping_records: list[dict[str, Any]],
|
||||||
assessment_package: dict[str, Any],
|
assessment_package: dict[str, Any],
|
||||||
|
export_manifest: dict[str, Any],
|
||||||
retention_summary: dict[str, Any],
|
retention_summary: dict[str, Any],
|
||||||
) -> None:
|
) -> None:
|
||||||
write_json(run_dir / "run.json", run_metadata)
|
write_json(run_dir / "run.json", run_metadata)
|
||||||
@@ -447,6 +465,8 @@ def _write_run_directory(
|
|||||||
write_json(run_dir / "normalized" / "findings.json", {"findings": findings})
|
write_json(run_dir / "normalized" / "findings.json", {"findings": findings})
|
||||||
write_json(run_dir / "normalized" / "mappings.json", {"mappings": mapping_records})
|
write_json(run_dir / "normalized" / "mappings.json", {"mappings": mapping_records})
|
||||||
write_json(run_dir / "reports" / "assessment-package.json", assessment_package)
|
write_json(run_dir / "reports" / "assessment-package.json", assessment_package)
|
||||||
|
write_json(run_dir / "reports" / "fragments.json", {"fragments": assessment_package["report_fragments"]})
|
||||||
|
write_json(run_dir / "exports" / "export-manifest.json", export_manifest)
|
||||||
(run_dir / "reports").mkdir(parents=True, exist_ok=True)
|
(run_dir / "reports").mkdir(parents=True, exist_ok=True)
|
||||||
(run_dir / "reports" / "report.md").write_text(
|
(run_dir / "reports" / "report.md").write_text(
|
||||||
_markdown_report(run_metadata, assessment_package),
|
_markdown_report(run_metadata, assessment_package),
|
||||||
@@ -471,6 +491,7 @@ def _markdown_report(run_metadata: dict[str, Any], package: dict[str, Any]) -> s
|
|||||||
mapping_lines = _mapping_summary_lines(package)
|
mapping_lines = _mapping_summary_lines(package)
|
||||||
policy_lines = _policy_summary_lines(package)
|
policy_lines = _policy_summary_lines(package)
|
||||||
review_lines = _review_summary_lines(package)
|
review_lines = _review_summary_lines(package)
|
||||||
|
fragment_lines = markdown_for_fragments(package.get("report_fragments", []))
|
||||||
|
|
||||||
return "\n".join(
|
return "\n".join(
|
||||||
[
|
[
|
||||||
@@ -496,6 +517,10 @@ def _markdown_report(run_metadata: dict[str, Any], package: dict[str, Any]) -> s
|
|||||||
"",
|
"",
|
||||||
review_lines,
|
review_lines,
|
||||||
"",
|
"",
|
||||||
|
"## Extension Fragments",
|
||||||
|
"",
|
||||||
|
fragment_lines,
|
||||||
|
"",
|
||||||
"## Boundary",
|
"## Boundary",
|
||||||
"",
|
"",
|
||||||
package["certification_boundary"],
|
package["certification_boundary"],
|
||||||
|
|||||||
50
src/guide_board/exports.py
Normal file
50
src/guide_board/exports.py
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
"""Portable export builders derived from assessment packages."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from guide_board.schema import assert_valid
|
||||||
|
|
||||||
|
|
||||||
|
def build_export_manifest(assessment_package: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
manifest = {
|
||||||
|
"id": f"export-manifest:{assessment_package['run_id']}",
|
||||||
|
"schema_version": "guide-board.export-manifest.v1",
|
||||||
|
"export_type": "guide-board.generic-json.v1",
|
||||||
|
"run_id": assessment_package["run_id"],
|
||||||
|
"created_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
"source_package_ref": "reports/assessment-package.json",
|
||||||
|
"source_lock_ref": "sources.lock.json",
|
||||||
|
"summary": assessment_package.get("summary", {}),
|
||||||
|
"policy_summary": assessment_package.get("policy_summary", {}),
|
||||||
|
"mapping_summary": assessment_package.get("mapping_summary", {}),
|
||||||
|
"report_fragments": [
|
||||||
|
_export_fragment(fragment)
|
||||||
|
for fragment in assessment_package.get("report_fragments", [])
|
||||||
|
if isinstance(fragment, dict)
|
||||||
|
],
|
||||||
|
"counts": {
|
||||||
|
"evidence_refs": len(assessment_package.get("evidence_refs", [])),
|
||||||
|
"findings": len(assessment_package.get("findings", [])),
|
||||||
|
"artifacts": len(assessment_package.get("artifact_manifest", [])),
|
||||||
|
"waivers": len(assessment_package.get("waivers", [])),
|
||||||
|
"challenges": len(assessment_package.get("challenges", [])),
|
||||||
|
"exclusions": len(assessment_package.get("exclusions", [])),
|
||||||
|
"report_fragments": len(assessment_package.get("report_fragments", [])),
|
||||||
|
},
|
||||||
|
"certification_boundary": assessment_package["certification_boundary"],
|
||||||
|
}
|
||||||
|
assert_valid(manifest, "export-manifest")
|
||||||
|
return manifest
|
||||||
|
|
||||||
|
|
||||||
|
def _export_fragment(fragment: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"id": fragment.get("id"),
|
||||||
|
"extension_id": fragment.get("extension_id"),
|
||||||
|
"title": fragment.get("title"),
|
||||||
|
"kind": fragment.get("kind"),
|
||||||
|
"structured": fragment.get("structured", {}),
|
||||||
|
}
|
||||||
204
src/guide_board/reports.py
Normal file
204
src/guide_board/reports.py
Normal file
@@ -0,0 +1,204 @@
|
|||||||
|
"""Report fragment loading for extension-contributed report content."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
from pathlib import Path
|
||||||
|
from types import ModuleType
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from guide_board.errors import ValidationError
|
||||||
|
from guide_board.io import load_json
|
||||||
|
|
||||||
|
|
||||||
|
def build_report_fragments(
|
||||||
|
root: Path,
|
||||||
|
run_dir: Path,
|
||||||
|
run_id: str,
|
||||||
|
plan: dict[str, Any],
|
||||||
|
evidence: list[dict[str, Any]],
|
||||||
|
findings: list[dict[str, Any]],
|
||||||
|
mapping_records: list[dict[str, Any]],
|
||||||
|
assessment_package: dict[str, Any],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
fragments: list[dict[str, Any]] = []
|
||||||
|
for extension in plan["extension_snapshots"]:
|
||||||
|
extension_path = _snapshot_path(root, extension)
|
||||||
|
manifest = load_json(extension_path / "extension.json")
|
||||||
|
for descriptor in manifest.get("report_fragments", []):
|
||||||
|
fragment = _load_fragment(
|
||||||
|
root,
|
||||||
|
run_dir,
|
||||||
|
run_id,
|
||||||
|
plan,
|
||||||
|
evidence,
|
||||||
|
findings,
|
||||||
|
mapping_records,
|
||||||
|
assessment_package,
|
||||||
|
extension,
|
||||||
|
extension_path,
|
||||||
|
descriptor,
|
||||||
|
)
|
||||||
|
if fragment is not None:
|
||||||
|
fragments.append(fragment)
|
||||||
|
return fragments
|
||||||
|
|
||||||
|
|
||||||
|
def markdown_for_fragments(fragments: list[dict[str, Any]]) -> str:
|
||||||
|
markdown_blocks = [
|
||||||
|
fragment.get("markdown", "")
|
||||||
|
for fragment in fragments
|
||||||
|
if isinstance(fragment.get("markdown"), str) and fragment.get("markdown")
|
||||||
|
]
|
||||||
|
if not markdown_blocks:
|
||||||
|
return "- no extension report fragments"
|
||||||
|
return "\n\n".join(markdown_blocks)
|
||||||
|
|
||||||
|
|
||||||
|
def _load_fragment(
|
||||||
|
root: Path,
|
||||||
|
run_dir: Path,
|
||||||
|
run_id: str,
|
||||||
|
plan: dict[str, Any],
|
||||||
|
evidence: list[dict[str, Any]],
|
||||||
|
findings: list[dict[str, Any]],
|
||||||
|
mapping_records: list[dict[str, Any]],
|
||||||
|
assessment_package: dict[str, Any],
|
||||||
|
extension: dict[str, Any],
|
||||||
|
extension_path: Path,
|
||||||
|
descriptor: Any,
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
if isinstance(descriptor, str):
|
||||||
|
descriptor = {
|
||||||
|
"id": descriptor,
|
||||||
|
"kind": "markdown_file",
|
||||||
|
"path": f"reports/{descriptor}.md",
|
||||||
|
"title": descriptor,
|
||||||
|
}
|
||||||
|
if not isinstance(descriptor, dict):
|
||||||
|
return None
|
||||||
|
|
||||||
|
if descriptor["kind"] == "markdown_file":
|
||||||
|
markdown = _load_markdown_fragment(extension_path, descriptor)
|
||||||
|
return _fragment_record(extension["id"], descriptor, markdown, {})
|
||||||
|
if descriptor["kind"] == "python_module":
|
||||||
|
result = _run_python_fragment(
|
||||||
|
root,
|
||||||
|
run_dir,
|
||||||
|
run_id,
|
||||||
|
plan,
|
||||||
|
evidence,
|
||||||
|
findings,
|
||||||
|
mapping_records,
|
||||||
|
assessment_package,
|
||||||
|
extension_path,
|
||||||
|
descriptor,
|
||||||
|
)
|
||||||
|
return _fragment_record(
|
||||||
|
extension["id"],
|
||||||
|
descriptor,
|
||||||
|
result.get("markdown", ""),
|
||||||
|
_object_or_empty(result.get("structured")),
|
||||||
|
)
|
||||||
|
raise ValidationError(f"{descriptor['id']}: unsupported report fragment kind")
|
||||||
|
|
||||||
|
|
||||||
|
def _load_markdown_fragment(extension_path: Path, descriptor: dict[str, Any]) -> str:
|
||||||
|
raw_path = descriptor.get("path") or f"reports/{descriptor['id']}.md"
|
||||||
|
fragment_path = _safe_extension_path(extension_path, raw_path, descriptor["id"])
|
||||||
|
if not fragment_path.is_file():
|
||||||
|
raise ValidationError(f"{descriptor['id']}: report fragment not found: {raw_path}")
|
||||||
|
return fragment_path.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def _run_python_fragment(
|
||||||
|
root: Path,
|
||||||
|
run_dir: Path,
|
||||||
|
run_id: str,
|
||||||
|
plan: dict[str, Any],
|
||||||
|
evidence: list[dict[str, Any]],
|
||||||
|
findings: list[dict[str, Any]],
|
||||||
|
mapping_records: list[dict[str, Any]],
|
||||||
|
assessment_package: dict[str, Any],
|
||||||
|
extension_path: Path,
|
||||||
|
descriptor: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
module_path = descriptor.get("module_path")
|
||||||
|
callable_name = descriptor.get("callable")
|
||||||
|
if not module_path or not callable_name:
|
||||||
|
raise ValidationError(
|
||||||
|
f"{descriptor['id']}: python_module report fragments need module_path and callable"
|
||||||
|
)
|
||||||
|
|
||||||
|
module_file = _safe_extension_path(extension_path, module_path, descriptor["id"])
|
||||||
|
module = _load_module(module_file, descriptor["id"])
|
||||||
|
fragment_callable = getattr(module, callable_name, None)
|
||||||
|
if not callable(fragment_callable):
|
||||||
|
raise ValidationError(f"{descriptor['id']}: callable {callable_name!r} was not found")
|
||||||
|
|
||||||
|
context = {
|
||||||
|
"root": str(root),
|
||||||
|
"run_dir": str(run_dir),
|
||||||
|
"run_id": run_id,
|
||||||
|
"plan": plan,
|
||||||
|
"evidence": evidence,
|
||||||
|
"findings": findings,
|
||||||
|
"mappings": mapping_records,
|
||||||
|
"assessment_package": assessment_package,
|
||||||
|
"policy_summary": assessment_package.get("policy_summary", {}),
|
||||||
|
"source_lock": assessment_package.get("source_lock", {}),
|
||||||
|
"extension_path": str(extension_path),
|
||||||
|
"report_fragment": descriptor,
|
||||||
|
}
|
||||||
|
result = fragment_callable(context)
|
||||||
|
if not isinstance(result, dict):
|
||||||
|
raise ValidationError(f"{descriptor['id']}: report fragment must return an object")
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def _fragment_record(
|
||||||
|
extension_id: str,
|
||||||
|
descriptor: dict[str, Any],
|
||||||
|
markdown: str,
|
||||||
|
structured: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"id": descriptor["id"],
|
||||||
|
"extension_id": extension_id,
|
||||||
|
"title": descriptor.get("title") or descriptor["id"],
|
||||||
|
"kind": descriptor["kind"],
|
||||||
|
"markdown": markdown if isinstance(markdown, str) else "",
|
||||||
|
"structured": structured,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_extension_path(extension_path: Path, raw_path: str, fragment_id: str) -> Path:
|
||||||
|
path = (extension_path / raw_path).resolve()
|
||||||
|
try:
|
||||||
|
path.relative_to(extension_path.resolve())
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ValidationError(
|
||||||
|
f"{fragment_id}: report fragment path must stay inside the extension directory"
|
||||||
|
) from exc
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def _load_module(path: Path, fragment_id: str) -> ModuleType:
|
||||||
|
if not path.exists():
|
||||||
|
raise ValidationError(f"{fragment_id}: module not found: {path}")
|
||||||
|
module_name = f"_guide_board_report_fragment_{fragment_id.replace('-', '_')}"
|
||||||
|
spec = importlib.util.spec_from_file_location(module_name, path)
|
||||||
|
if spec is None or spec.loader is None:
|
||||||
|
raise ValidationError(f"{fragment_id}: unable to load module from {path}")
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
def _snapshot_path(root: Path, extension: dict[str, Any]) -> Path:
|
||||||
|
path = Path(extension["path"])
|
||||||
|
return path if path.is_absolute() else root / path
|
||||||
|
|
||||||
|
|
||||||
|
def _object_or_empty(value: Any) -> dict[str, Any]:
|
||||||
|
return value if isinstance(value, dict) else {}
|
||||||
@@ -49,8 +49,12 @@ def build_retention_summary(
|
|||||||
"report_refs": [
|
"report_refs": [
|
||||||
"reports/assessment-package.json",
|
"reports/assessment-package.json",
|
||||||
"reports/report.md",
|
"reports/report.md",
|
||||||
|
"reports/fragments.json",
|
||||||
"reports/submission-package.json",
|
"reports/submission-package.json",
|
||||||
],
|
],
|
||||||
|
"export_refs": [
|
||||||
|
"exports/export-manifest.json",
|
||||||
|
],
|
||||||
"artifact_retention": {
|
"artifact_retention": {
|
||||||
"policy": plan["assessment_profile_snapshot"].get("retention_policy", {}),
|
"policy": plan["assessment_profile_snapshot"].get("retention_policy", {}),
|
||||||
"output_artifact_retention": plan["assessment_profile_snapshot"]
|
"output_artifact_retention": plan["assessment_profile_snapshot"]
|
||||||
@@ -200,6 +204,7 @@ def _run_projection(run: dict[str, Any]) -> dict[str, Any]:
|
|||||||
"status": summary.get("status", "unknown"),
|
"status": summary.get("status", "unknown"),
|
||||||
"unexpected_findings": _summary_int(summary, "unexpected_findings"),
|
"unexpected_findings": _summary_int(summary, "unexpected_findings"),
|
||||||
"finding_count": _summary_int(summary, "finding_count"),
|
"finding_count": _summary_int(summary, "finding_count"),
|
||||||
|
"mapping_target_count": _summary_int(summary, "mapping_target_count"),
|
||||||
"artifact_count": _summary_int(summary, "artifact_count"),
|
"artifact_count": _summary_int(summary, "artifact_count"),
|
||||||
"challenged_findings": _summary_int(summary, "challenged_findings"),
|
"challenged_findings": _summary_int(summary, "challenged_findings"),
|
||||||
"authority_exclusions": _summary_int(summary, "authority_exclusions"),
|
"authority_exclusions": _summary_int(summary, "authority_exclusions"),
|
||||||
@@ -217,12 +222,18 @@ def _trend_between(
|
|||||||
return {
|
return {
|
||||||
"direction": "insufficient-history",
|
"direction": "insufficient-history",
|
||||||
"status_changed": False,
|
"status_changed": False,
|
||||||
|
"status_change": {
|
||||||
|
"from": None,
|
||||||
|
"to": _status_for(latest),
|
||||||
|
},
|
||||||
"unexpected_findings_delta": 0,
|
"unexpected_findings_delta": 0,
|
||||||
"finding_count_delta": 0,
|
"finding_count_delta": 0,
|
||||||
"artifact_count_delta": 0,
|
"artifact_count_delta": 0,
|
||||||
"unresolved_review_items_delta": 0,
|
"unresolved_review_items_delta": 0,
|
||||||
"evidence_result_deltas": {},
|
"mapping_target_count_delta": 0,
|
||||||
}
|
"evidence_result_deltas": {},
|
||||||
|
"summary_text": "No previous retained run is available for comparison.",
|
||||||
|
}
|
||||||
|
|
||||||
previous_summary = previous.get("summary", {})
|
previous_summary = previous.get("summary", {})
|
||||||
latest_summary = latest.get("summary", {})
|
latest_summary = latest.get("summary", {})
|
||||||
@@ -242,17 +253,34 @@ def _trend_between(
|
|||||||
review_delta = _summary_int(latest_summary, "unresolved_review_items") - _summary_int(
|
review_delta = _summary_int(latest_summary, "unresolved_review_items") - _summary_int(
|
||||||
previous_summary, "unresolved_review_items"
|
previous_summary, "unresolved_review_items"
|
||||||
)
|
)
|
||||||
|
mapping_target_delta = _summary_int(latest_summary, "mapping_target_count") - _summary_int(
|
||||||
|
previous_summary, "mapping_target_count"
|
||||||
|
)
|
||||||
previous_status = _status_for(previous)
|
previous_status = _status_for(previous)
|
||||||
latest_status = _status_for(latest)
|
latest_status = _status_for(latest)
|
||||||
|
direction = _trend_direction(previous_status, latest_status, unexpected_delta)
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"direction": _trend_direction(previous_status, latest_status, unexpected_delta),
|
"direction": direction,
|
||||||
"status_changed": previous_status != latest_status,
|
"status_changed": previous_status != latest_status,
|
||||||
|
"status_change": {
|
||||||
|
"from": previous_status,
|
||||||
|
"to": latest_status,
|
||||||
|
},
|
||||||
"unexpected_findings_delta": unexpected_delta,
|
"unexpected_findings_delta": unexpected_delta,
|
||||||
"finding_count_delta": finding_delta,
|
"finding_count_delta": finding_delta,
|
||||||
"artifact_count_delta": artifact_delta,
|
"artifact_count_delta": artifact_delta,
|
||||||
"unresolved_review_items_delta": review_delta,
|
"unresolved_review_items_delta": review_delta,
|
||||||
|
"mapping_target_count_delta": mapping_target_delta,
|
||||||
"evidence_result_deltas": evidence_deltas,
|
"evidence_result_deltas": evidence_deltas,
|
||||||
|
"summary_text": _trend_summary_text(
|
||||||
|
direction,
|
||||||
|
previous_status,
|
||||||
|
latest_status,
|
||||||
|
unexpected_delta,
|
||||||
|
review_delta,
|
||||||
|
mapping_target_delta,
|
||||||
|
),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -294,6 +322,24 @@ def _summary_int(summary: dict[str, Any], key: str) -> int:
|
|||||||
return value if isinstance(value, int) and not isinstance(value, bool) else 0
|
return value if isinstance(value, int) and not isinstance(value, bool) else 0
|
||||||
|
|
||||||
|
|
||||||
|
def _trend_summary_text(
|
||||||
|
direction: str,
|
||||||
|
previous_status: str,
|
||||||
|
latest_status: str,
|
||||||
|
unexpected_delta: int,
|
||||||
|
review_delta: int,
|
||||||
|
mapping_target_delta: int,
|
||||||
|
) -> str:
|
||||||
|
parts = [
|
||||||
|
f"Trend {direction}",
|
||||||
|
f"status {previous_status} -> {latest_status}",
|
||||||
|
f"unexpected findings delta {unexpected_delta}",
|
||||||
|
f"unresolved review delta {review_delta}",
|
||||||
|
f"mapping target delta {mapping_target_delta}",
|
||||||
|
]
|
||||||
|
return "; ".join(parts) + "."
|
||||||
|
|
||||||
|
|
||||||
def _dict_deltas(previous: Any, latest: Any) -> dict[str, int]:
|
def _dict_deltas(previous: Any, latest: Any) -> dict[str, int]:
|
||||||
previous_dict = previous if isinstance(previous, dict) else {}
|
previous_dict = previous if isinstance(previous, dict) else {}
|
||||||
latest_dict = latest if isinstance(latest, dict) else {}
|
latest_dict = latest if isinstance(latest, dict) else {}
|
||||||
|
|||||||
@@ -285,6 +285,12 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler):
|
|||||||
if isinstance(submission_value, str) and submission_value
|
if isinstance(submission_value, str) and submission_value
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
|
export_value = result.get("export_manifest")
|
||||||
|
export_path = (
|
||||||
|
Path(export_value)
|
||||||
|
if isinstance(export_value, str) and export_value
|
||||||
|
else None
|
||||||
|
)
|
||||||
try:
|
try:
|
||||||
report_markdown = report_path.read_text(encoding="utf-8")
|
report_markdown = report_path.read_text(encoding="utf-8")
|
||||||
assessment_package = load_json(package_path)
|
assessment_package = load_json(package_path)
|
||||||
@@ -294,6 +300,11 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler):
|
|||||||
if submission_path is not None and submission_path.is_file()
|
if submission_path is not None and submission_path.is_file()
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
|
export_manifest = (
|
||||||
|
load_json(export_path)
|
||||||
|
if export_path is not None and export_path.is_file()
|
||||||
|
else None
|
||||||
|
)
|
||||||
except OSError as exc:
|
except OSError as exc:
|
||||||
raise HttpProblem(404, f"run report artifact is missing: {exc}") from exc
|
raise HttpProblem(404, f"run report artifact is missing: {exc}") from exc
|
||||||
|
|
||||||
@@ -307,6 +318,7 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler):
|
|||||||
"assessment_package": str(package_path),
|
"assessment_package": str(package_path),
|
||||||
"retention_summary": str(retention_path),
|
"retention_summary": str(retention_path),
|
||||||
"submission_package": str(submission_path) if submission_path is not None else None,
|
"submission_package": str(submission_path) if submission_path is not None else None,
|
||||||
|
"export_manifest": str(export_path) if export_path is not None else None,
|
||||||
},
|
},
|
||||||
"report": {
|
"report": {
|
||||||
"path": str(report_path),
|
"path": str(report_path),
|
||||||
@@ -324,6 +336,10 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler):
|
|||||||
"path": str(submission_path) if submission_package else None,
|
"path": str(submission_path) if submission_package else None,
|
||||||
"json": submission_package,
|
"json": submission_package,
|
||||||
},
|
},
|
||||||
|
"export_manifest": {
|
||||||
|
"path": str(export_path) if export_manifest else None,
|
||||||
|
"json": export_manifest,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
def _retained_runs(self, query: dict[str, str]) -> dict[str, Any]:
|
def _retained_runs(self, query: dict[str, str]) -> dict[str, Any]:
|
||||||
|
|||||||
17
tests/golden/export-manifest-shape.json
Normal file
17
tests/golden/export-manifest-shape.json
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
{
|
||||||
|
"top_level_keys": [
|
||||||
|
"certification_boundary",
|
||||||
|
"counts",
|
||||||
|
"created_at",
|
||||||
|
"export_type",
|
||||||
|
"id",
|
||||||
|
"mapping_summary",
|
||||||
|
"policy_summary",
|
||||||
|
"report_fragments",
|
||||||
|
"run_id",
|
||||||
|
"schema_version",
|
||||||
|
"source_lock_ref",
|
||||||
|
"source_package_ref",
|
||||||
|
"summary"
|
||||||
|
]
|
||||||
|
}
|
||||||
5
tests/golden/sdk-fixture-report-fragment.md
Normal file
5
tests/golden/sdk-fixture-report-fragment.md
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
### SDK Fixture Summary
|
||||||
|
|
||||||
|
- evidence items: 2
|
||||||
|
- findings: 0
|
||||||
|
- source lock: source-lock:sdk-fixture-assessment:sdk-fixture-target
|
||||||
@@ -230,6 +230,10 @@ class CoreArchitectureTests(unittest.TestCase):
|
|||||||
assessment_package = json.loads(
|
assessment_package = json.loads(
|
||||||
(run_dir / "reports" / "assessment-package.json").read_text(encoding="utf-8")
|
(run_dir / "reports" / "assessment-package.json").read_text(encoding="utf-8")
|
||||||
)
|
)
|
||||||
|
report = (run_dir / "reports" / "report.md").read_text(encoding="utf-8")
|
||||||
|
export_manifest = json.loads(
|
||||||
|
(run_dir / "exports" / "export-manifest.json").read_text(encoding="utf-8")
|
||||||
|
)
|
||||||
|
|
||||||
self.assertEqual(result["status"], "completed")
|
self.assertEqual(result["status"], "completed")
|
||||||
self.assertEqual(plan["extension_snapshots"][0]["source"], "external")
|
self.assertEqual(plan["extension_snapshots"][0]["source"], "external")
|
||||||
@@ -259,6 +263,21 @@ class CoreArchitectureTests(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
self.assertEqual(mappings[0]["target_id"], "normalizer-plugin")
|
self.assertEqual(mappings[0]["target_id"], "normalizer-plugin")
|
||||||
self.assertEqual(assessment_package["summary"], {"pass": 1, "skipped": 1})
|
self.assertEqual(assessment_package["summary"], {"pass": 1, "skipped": 1})
|
||||||
|
self.assertEqual(
|
||||||
|
assessment_package["report_fragments"][0]["markdown"],
|
||||||
|
(ROOT / "tests" / "golden" / "sdk-fixture-report-fragment.md")
|
||||||
|
.read_text(encoding="utf-8")
|
||||||
|
.rstrip(),
|
||||||
|
)
|
||||||
|
self.assertIn("### SDK Fixture Summary", report)
|
||||||
|
assert_valid(export_manifest, "export-manifest")
|
||||||
|
export_shape = load_json(ROOT / "tests" / "golden" / "export-manifest-shape.json")
|
||||||
|
self.assertEqual(sorted(export_manifest), export_shape["top_level_keys"])
|
||||||
|
self.assertEqual(export_manifest["counts"]["report_fragments"], 1)
|
||||||
|
self.assertEqual(
|
||||||
|
export_manifest["report_fragments"][0]["structured"]["evidence_count"],
|
||||||
|
2,
|
||||||
|
)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
assessment_package["source_lock"]["metadata_hooks"]["runner_entrypoints"][0][
|
assessment_package["source_lock"]["metadata_hooks"]["runner_entrypoints"][0][
|
||||||
"metadata"
|
"metadata"
|
||||||
@@ -297,7 +316,9 @@ class CoreArchitectureTests(unittest.TestCase):
|
|||||||
self.assertTrue((run_dir / "normalized" / "evidence.json").exists())
|
self.assertTrue((run_dir / "normalized" / "evidence.json").exists())
|
||||||
self.assertTrue((run_dir / "reports" / "assessment-package.json").exists())
|
self.assertTrue((run_dir / "reports" / "assessment-package.json").exists())
|
||||||
self.assertTrue((run_dir / "reports" / "report.md").exists())
|
self.assertTrue((run_dir / "reports" / "report.md").exists())
|
||||||
|
self.assertTrue((run_dir / "reports" / "fragments.json").exists())
|
||||||
self.assertTrue((run_dir / "reports" / "submission-package.json").exists())
|
self.assertTrue((run_dir / "reports" / "submission-package.json").exists())
|
||||||
|
self.assertTrue((run_dir / "exports" / "export-manifest.json").exists())
|
||||||
retention = json.loads(
|
retention = json.loads(
|
||||||
(run_dir / "retention-summary.json").read_text(encoding="utf-8")
|
(run_dir / "retention-summary.json").read_text(encoding="utf-8")
|
||||||
)
|
)
|
||||||
@@ -309,9 +330,14 @@ class CoreArchitectureTests(unittest.TestCase):
|
|||||||
result["submission_package"],
|
result["submission_package"],
|
||||||
str(run_dir / "reports" / "submission-package.json"),
|
str(run_dir / "reports" / "submission-package.json"),
|
||||||
)
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
result["export_manifest"],
|
||||||
|
str(run_dir / "exports" / "export-manifest.json"),
|
||||||
|
)
|
||||||
self.assertEqual(retention["summary"]["status"], "completed")
|
self.assertEqual(retention["summary"]["status"], "completed")
|
||||||
self.assertEqual(retention["summary"]["artifact_count"], 0)
|
self.assertEqual(retention["summary"]["artifact_count"], 0)
|
||||||
self.assertIn("reports/submission-package.json", retention["report_refs"])
|
self.assertIn("reports/submission-package.json", retention["report_refs"])
|
||||||
|
self.assertIn("exports/export-manifest.json", retention["export_refs"])
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
retention["artifact_retention"]["policy"],
|
retention["artifact_retention"]["policy"],
|
||||||
{"raw_artifact_days": 0, "summary_days": 365},
|
{"raw_artifact_days": 0, "summary_days": 365},
|
||||||
@@ -463,6 +489,10 @@ class CoreArchitectureTests(unittest.TestCase):
|
|||||||
reports["submission_package"]["json"]["run_id"],
|
reports["submission_package"]["json"]["run_id"],
|
||||||
status["result"]["run_id"],
|
status["result"]["run_id"],
|
||||||
)
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
reports["export_manifest"]["json"]["run_id"],
|
||||||
|
status["result"]["run_id"],
|
||||||
|
)
|
||||||
finally:
|
finally:
|
||||||
service.stop()
|
service.stop()
|
||||||
|
|
||||||
@@ -552,7 +582,13 @@ class CoreArchitectureTests(unittest.TestCase):
|
|||||||
self.assertEqual(group["previous_run"]["run_id"], "run-old")
|
self.assertEqual(group["previous_run"]["run_id"], "run-old")
|
||||||
self.assertEqual(group["trend"]["direction"], "improved")
|
self.assertEqual(group["trend"]["direction"], "improved")
|
||||||
self.assertTrue(group["trend"]["status_changed"])
|
self.assertTrue(group["trend"]["status_changed"])
|
||||||
|
self.assertEqual(
|
||||||
|
group["trend"]["status_change"],
|
||||||
|
{"from": "blocked", "to": "completed"},
|
||||||
|
)
|
||||||
self.assertEqual(group["trend"]["unexpected_findings_delta"], -1)
|
self.assertEqual(group["trend"]["unexpected_findings_delta"], -1)
|
||||||
|
self.assertEqual(group["trend"]["mapping_target_count_delta"], 0)
|
||||||
|
self.assertIn("Trend improved", group["trend"]["summary_text"])
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
group["trend"]["evidence_result_deltas"],
|
group["trend"]["evidence_result_deltas"],
|
||||||
{"blocked": -1, "manual": 1, "skipped": 1},
|
{"blocked": -1, "manual": 1, "skipped": 1},
|
||||||
|
|||||||
@@ -4,12 +4,12 @@ type: workplan
|
|||||||
title: "Report And Export Maturity"
|
title: "Report And Export Maturity"
|
||||||
repo: guide-board
|
repo: guide-board
|
||||||
domain: markitect
|
domain: markitect
|
||||||
status: active
|
status: completed
|
||||||
owner: codex
|
owner: codex
|
||||||
planning_priority: medium
|
planning_priority: medium
|
||||||
planning_order: 7
|
planning_order: 7
|
||||||
created: "2026-05-15"
|
created: "2026-05-15"
|
||||||
updated: "2026-05-15"
|
updated: "2026-05-16"
|
||||||
state_hub_workstream_id: "ef9351d2-e99c-470e-aeec-f17aa51eae14"
|
state_hub_workstream_id: "ef9351d2-e99c-470e-aeec-f17aa51eae14"
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -41,7 +41,7 @@ extension-provided fragments or exporters.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: GUIDE-BOARD-WP-0007-T001
|
id: GUIDE-BOARD-WP-0007-T001
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "bf3fe163-b06d-4c2e-9b45-31721864e1f2"
|
state_hub_task_id: "bf3fe163-b06d-4c2e-9b45-31721864e1f2"
|
||||||
```
|
```
|
||||||
@@ -55,11 +55,19 @@ Acceptance:
|
|||||||
summary, and source lock data.
|
summary, and source lock data.
|
||||||
- Add a fixture fragment and tests.
|
- Add a fixture fragment and tests.
|
||||||
|
|
||||||
|
Progress:
|
||||||
|
|
||||||
|
- Added Markdown file and Python module report fragment descriptors.
|
||||||
|
- Loaded fragment paths safely from extension roots.
|
||||||
|
- Added fragment context for assessment package, evidence, findings, mappings,
|
||||||
|
policy summary, and source lock data.
|
||||||
|
- Added an SDK fixture Python report fragment and focused tests.
|
||||||
|
|
||||||
## D7.2 - Portable Export Formats
|
## D7.2 - Portable Export Formats
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: GUIDE-BOARD-WP-0007-T002
|
id: GUIDE-BOARD-WP-0007-T002
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "fda51e62-98aa-408e-a057-4db40fe7c644"
|
state_hub_task_id: "fda51e62-98aa-408e-a057-4db40fe7c644"
|
||||||
```
|
```
|
||||||
@@ -72,11 +80,20 @@ Acceptance:
|
|||||||
- Preserve certification boundary and source lock references in each export.
|
- Preserve certification boundary and source lock references in each export.
|
||||||
- Document which exports are generic and which must remain extension-owned.
|
- Document which exports are generic and which must remain extension-owned.
|
||||||
|
|
||||||
|
Progress:
|
||||||
|
|
||||||
|
- Added `docs/schemas/export-manifest.schema.json`.
|
||||||
|
- Wrote `exports/export-manifest.json` for each run.
|
||||||
|
- Included source package refs, source lock refs, summaries, policy summaries,
|
||||||
|
mapping summaries, report fragments, counts, and certification boundary.
|
||||||
|
- Documented the generic export boundary; authority-specific formats remain
|
||||||
|
extension-owned.
|
||||||
|
|
||||||
## D7.3 - Trend And Gate Report Improvements
|
## D7.3 - Trend And Gate Report Improvements
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: GUIDE-BOARD-WP-0007-T003
|
id: GUIDE-BOARD-WP-0007-T003
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "33c3089a-9d5e-4605-89c4-a1e070bc12ad"
|
state_hub_task_id: "33c3089a-9d5e-4605-89c4-a1e070bc12ad"
|
||||||
```
|
```
|
||||||
@@ -89,11 +106,19 @@ Acceptance:
|
|||||||
- Keep machine-readable gate summaries stable for automation.
|
- Keep machine-readable gate summaries stable for automation.
|
||||||
- Add CLI report helpers or Markdown summaries where useful.
|
- Add CLI report helpers or Markdown summaries where useful.
|
||||||
|
|
||||||
|
Progress:
|
||||||
|
|
||||||
|
- Added trend status-change details, unresolved review deltas, mapping target
|
||||||
|
deltas, and compact summary text.
|
||||||
|
- Kept existing gate summary shape stable while improving trend inputs for
|
||||||
|
human review.
|
||||||
|
- Added assertions for the richer trend output.
|
||||||
|
|
||||||
## D7.4 - Golden Fixtures And Documentation
|
## D7.4 - Golden Fixtures And Documentation
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: GUIDE-BOARD-WP-0007-T004
|
id: GUIDE-BOARD-WP-0007-T004
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "66669f68-6728-4484-9ec9-267ffe025027"
|
state_hub_task_id: "66669f68-6728-4484-9ec9-267ffe025027"
|
||||||
```
|
```
|
||||||
@@ -106,6 +131,15 @@ Acceptance:
|
|||||||
- Ensure report text remains clear about preparation evidence versus formal
|
- Ensure report text remains clear about preparation evidence versus formal
|
||||||
certification.
|
certification.
|
||||||
|
|
||||||
|
Progress:
|
||||||
|
|
||||||
|
- Added golden fixture expectations for report fragment text and export manifest
|
||||||
|
top-level shape.
|
||||||
|
- Updated extension SDK, assessment operations, architecture, service, container,
|
||||||
|
and README docs.
|
||||||
|
- Kept report and export boundary language tied to preparation evidence, not
|
||||||
|
formal certification.
|
||||||
|
|
||||||
## Definition Of Done
|
## Definition Of Done
|
||||||
|
|
||||||
- Extensions can contribute report fragments through a documented contract.
|
- Extensions can contribute report fragments through a documented contract.
|
||||||
|
|||||||
Reference in New Issue
Block a user