coulomb/guide-board
2
0
Fork 0
generated from coulomb/repo-seed
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement source lock and submission package baseline

Browse Source
This commit is contained in:
Bernd Worsch
2026-05-16 02:51:00 +02:00
parent d73a73b455
commit c8ac42154c
18 changed files with 852 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@@ -42,6 +42,9 @@ The same CLI contracts are packaged by the container baseline. See
[docs/CONTAINER.md](docs/CONTAINER.md). The dependency-light local API wraps
those contracts for service and container operation; see
[docs/LOCAL-SERVICE-API.md](docs/LOCAL-SERVICE-API.md).
New runs write a richer `sources.lock.json` and
`reports/submission-package.json` alongside the assessment package so reviewers
can inspect source, metadata, artifact, and boundary references.
The `sample-noop` extension exercises the guide-board contracts without invoking
an external harness. `sdk-fixture` demonstrates the extension SDK contracts for

37
docs/ARCHITECTURE-BLUEPRINT.md
View File

@@ -355,7 +355,9 @@ Stores run artifacts by reference and checksum:
The first implementation builds the assessment package artifact manifest from
runner-emitted artifact refs and computes checksums for files inside the run
directory.
directory. New runs also write a source lock and a submission package manifest
that fingerprint reviewable run files and summarize runner or normalizer
metadata reported by extensions.
### Normalizer
@@ -559,6 +561,18 @@ building complex runtime code.
- `artifact_policy`
- `runtime_policy`
### `SourceLock`
- `framework_refs`
- `extension_refs`
- `frameworks`
- `extensions`
- `mapping_sets`
- `profiles`
- `policy_refs`
- `authorities`
- `metadata_hooks`
### `RawArtifact`
- `id`
@@ -626,6 +640,19 @@ building complex runtime code.
- `certification_boundary`
- `created_at`
### `SubmissionPackage`
- `run_id`
- `package_identity`
- `source_lock_ref`
- `source_lock`
- `reports`
- `normalized_outputs`
- `profile_snapshots`
- `artifact_manifest`
- `reported_metadata`
- `certification_boundary`
## Result Vocabulary
The evidence model should allow these statuses:
@@ -714,6 +741,7 @@ runs/<run-id>/
reports/
report.md
assessment-package.json
submission-package.json
exports/
```
@@ -787,7 +815,12 @@ Each run should lock:
- test suite IDs,
- mapping version,
- target profile snapshot,
- waiver snapshot.
- expectation and waiver refs.
The current source lock remains backward-compatible with the original
`framework_refs` and `extension_refs` fields while adding checksummed profiles,
mapping-set refs, optional policy refs, authority descriptors, and metadata
hooks for runners and normalizers.
## Implementation Sequence

13
docs/ASSESSMENT-OPERATIONS.md
View File

@@ -77,6 +77,7 @@ A completed CLI command prints a JSON result with:
- `run_dir`: output directory,
- `assessment_package`: JSON assessment package path,
- `report`: Markdown report path,
- `submission_package`: portable submission package manifest path,
- `retention_summary`: compact durable summary path.
The output directory uses this contract:
@@ -84,15 +85,27 @@ The output directory uses this contract:
```text
run.json
plan.json
sources.lock.json
target-profile.snapshot.json
assessment-profile.snapshot.json
retention-summary.json
normalized/evidence.json
normalized/findings.json
normalized/mappings.json
reports/assessment-package.json
reports/report.md
reports/submission-package.json
artifacts/
```
`sources.lock.json` records the framework refs, extension versions, mapping
sets, profile snapshots, policy refs, authority refs, and extension metadata
hooks used for the run. `reports/submission-package.json` points at the
reviewable package files, includes checksums where files exist, carries the raw
artifact manifest, and repeats the certification boundary. It is a portable
handoff manifest for preparation evidence, not an authority-specific final
submission.
Use the retained run helpers for history:
```sh

44
docs/EXTENSION-SDK.md
View File

@@ -71,7 +71,12 @@ The key runtime fields are:
- `extension_type`: one of the supported archetypes from the architecture
blueprint.
- `supported_frameworks`: framework IDs this extension can contribute evidence
for.
for. Descriptor objects with `id`, `version`, `source_url`, and
`authority_ref` may be used when source metadata is available.
- `authorities`: authority IDs or descriptor objects with optional source URL,
version, license, and access notes.
- `metadata`: optional extension-level metadata such as adapter version or
source URL. The core preserves it in source locks and evidence metadata.
- `check_groups`: named groups that assessment profiles can select.
- `preflight_runner`: optional runner ID used before selected check groups.
- `runner_entrypoints`: concrete runner declarations.
@@ -141,6 +146,11 @@ Example:
"module_path": "src/open_cmis_tck/preflight.py",
"callable": "run",
"command": null,
"metadata": {
"harness_id": "opencmis-tck",
"harness_version": "extension-detected-or-declared",
"source_url": "https://chemistry.apache.org/java/opencmis.html"
},
"description": "Checks whether the CMIS Browser Binding endpoint is reachable."
}
```
@@ -272,11 +282,20 @@ Result fields:
- `observations`: human-readable observations.
- `facts`: structured facts extracted by the runner.
- `artifact_refs`: references to raw artifacts written by the runner.
- `requirement_refs`: optional requirement refs discovered by the runner.
- `metadata`: optional generic metadata such as `harness_version`,
`test_suite_id`, `adapter_version`, `source_url`, or native result IDs.
Artifact refs must be paths relative to the run directory. After runner
execution, the core fingerprints existing artifact refs into the assessment
package `artifact_manifest`.
Runner metadata is merged with manifest entrypoint metadata and preserved under
evidence `facts.source_metadata`. The same metadata is also summarized in the
submission package manifest, which lets reviewers distinguish the extension
version from the harness or native test-suite version without adding
domain-specific fields to the core.
If a Python runner raises an exception, the core converts that failure into
`infrastructure_error` evidence so the assessment package remains complete.
@@ -298,6 +317,9 @@ extension can add a normalizer descriptor:
"module_path": "normalizers/native_probe.py",
"callable": "normalize",
"runner_ref": "native-probe",
"metadata": {
"adapter_version": "0.1.0"
},
"description": "Converts native runner output into guide-board evidence."
}
```
@@ -340,6 +362,7 @@ The core merges the normalizer output over the runner result:
- `observations` are appended.
- `facts` are merged.
- `artifact_refs` and `requirement_refs` are deduplicated.
- `metadata` is merged.
- `normalizer_refs` is recorded in evidence facts when any normalizer runs.
If a normalizer raises an exception, the step becomes
@@ -350,6 +373,25 @@ The bundled `extensions/sdk-fixture` extension is the copyable reference path
for profile schemas, a native-output runner, a normalizer, mappings, and fixture
profiles.
## Source Lock And Submission Package
Every new run writes `sources.lock.json` and
`reports/submission-package.json`. Extension authors should treat source
metadata as part of the evidence contract:
- declare extension, authority, framework, runner, and normalizer metadata in
`extension.json` when it is static;
- return runner or normalizer `metadata` when versions, native result IDs, or
test-suite IDs are detected at runtime;
- keep mapping sets under `mappings/` so the core can checksum them in the
source lock;
- keep restricted or licensed assets referenced by metadata or artifacts rather
than vendored into the core.
The submission package manifest is generic guide-board output. Authority-specific
final submissions, trademark assertions, or certification conclusions remain
extension-owned or reviewer-owned.
## Result Statuses
Initial statuses:

36
docs/schemas/extension-manifest.schema.json
View File

@@ -41,8 +41,38 @@
"type": "string",
"enum": ["candidate", "incubating", "active", "external", "deprecated"]
},
"supported_frameworks": { "type": "array", "items": { "type": "string" } },
"authorities": { "type": "array", "items": { "type": "string" } },
"supported_frameworks": {
"type": "array",
"items": {
"type": ["string", "object"],
"additionalProperties": false,
"required": ["id"],
"properties": {
"id": { "type": "string" },
"version": { "type": ["string", "null"] },
"source_url": { "type": ["string", "null"] },
"authority_ref": { "type": ["string", "null"] },
"description": { "type": ["string", "null"] }
}
}
},
"authorities": {
"type": "array",
"items": {
"type": ["string", "object"],
"additionalProperties": false,
"required": ["id"],
"properties": {
"id": { "type": "string" },
"name": { "type": ["string", "null"] },
"version": { "type": ["string", "null"] },
"source_url": { "type": ["string", "null"] },
"license": { "type": ["string", "null"] },
"access": { "type": ["string", "null"] }
}
}
},
"metadata": { "type": "object" },
"profile_schemas": {
"type": "array",
"items": {
@@ -89,6 +119,7 @@
"module_path": { "type": ["string", "null"] },
"callable": { "type": ["string", "null"] },
"command": { "type": ["array", "null"], "items": { "type": "string" } },
"metadata": { "type": "object" },
"description": { "type": ["string", "null"] }
}
}
@@ -105,6 +136,7 @@
"module_path": { "type": "string" },
"callable": { "type": "string" },
"runner_ref": { "type": ["string", "null"] },
"metadata": { "type": "object" },
"description": { "type": ["string", "null"] }
}
}

34
docs/schemas/source-lock.schema.json Normal file
View File

@@ -0,0 +1,34 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "Guide Board Source Lock",
"type": "object",
"additionalProperties": false,
"required": [
"id",
"schema_version",
"created_at",
"framework_refs",
"extension_refs",
"frameworks",
"extensions",
"mapping_sets",
"profiles",
"policy_refs",
"authorities",
"metadata_hooks"
],
"properties": {
"id": { "type": "string" },
"schema_version": { "type": "string" },
"created_at": { "type": "string" },
"framework_refs": { "type": "array", "items": { "type": "string" } },
"extension_refs": { "type": "array", "items": { "type": "string" } },
"frameworks": { "type": "array", "items": { "type": "object" } },
"extensions": { "type": "array", "items": { "type": "object" } },
"mapping_sets": { "type": "array", "items": { "type": "object" } },
"profiles": { "type": "object" },
"policy_refs": { "type": "object" },
"authorities": { "type": "array", "items": { "type": "object" } },
"metadata_hooks": { "type": "object" }
}
}

36
docs/schemas/submission-package.schema.json Normal file
View File

@@ -0,0 +1,36 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "Guide Board Submission Package Manifest",
"type": "object",
"additionalProperties": false,
"required": [
"id",
"schema_version",
"run_id",
"created_at",
"package_identity",
"source_lock_ref",
"source_lock",
"reports",
"normalized_outputs",
"profile_snapshots",
"artifact_manifest",
"reported_metadata",
"certification_boundary"
],
"properties": {
"id": { "type": "string" },
"schema_version": { "type": "string" },
"run_id": { "type": "string" },
"created_at": { "type": "string" },
"package_identity": { "type": "object" },
"source_lock_ref": { "type": "string" },
"source_lock": { "type": "object" },
"reports": { "type": "array", "items": { "type": "object" } },
"normalized_outputs": { "type": "array", "items": { "type": "object" } },
"profile_snapshots": { "type": "array", "items": { "type": "object" } },
"artifact_manifest": { "type": "array", "items": { "type": "object" } },
"reported_metadata": { "type": "array", "items": { "type": "object" } },
"certification_boundary": { "type": "string" }
}
}

14
extensions/sdk-fixture/extension.json
View File

@@ -8,6 +8,10 @@
"guide-board.sdk-fixture.v1"
],
"authorities": [],
"metadata": {
"adapter_version": "0.1.0",
"source_url": "https://example.invalid/guide-board/sdk-fixture"
},
"profile_schemas": [
"target-profile",
"assessment-profile",
@@ -44,6 +48,12 @@
"module_path": "runners/native_probe.py",
"callable": "run",
"command": null,
"metadata": {
"harness_id": "sdk-fixture-native-probe",
"harness_version": "1.0.0",
"test_suite_id": "sdk-fixture-suite-v1",
"source_url": "https://example.invalid/guide-board/sdk-fixture/native-probe"
},
"description": "Writes a tiny native result artifact for the SDK fixture normalizer."
}
],
@@ -54,6 +64,10 @@
"module_path": "normalizers/native_probe.py",
"callable": "normalize",
"runner_ref": "native-probe",
"metadata": {
"adapter_version": "0.1.0",
"source_url": "https://example.invalid/guide-board/sdk-fixture/native-normalizer"
},
"description": "Converts the SDK fixture native result artifact into guide-board evidence."
}
],

4
extensions/sdk-fixture/normalizers/native_probe.py
View File

@@ -25,4 +25,8 @@ def normalize(context: dict) -> dict:
artifact_ref
],
"requirement_refs": native_result.get("requirement_refs", []),
"metadata": {
"normalizer_id": "native-probe-normalizer",
"native_result_id": "sdk-fixture-native-result"
},
}

4
extensions/sdk-fixture/runners/native_probe.py
View File

@@ -33,4 +33,8 @@ def run(context: dict) -> dict:
"artifact_refs": [
artifact_ref
],
"metadata": {
"native_result_id": "sdk-fixture-native-result",
"test_suite_id": "sdk-fixture-suite-v1"
},
}

101
src/guide_board/artifacts.py
View File

@@ -46,6 +46,107 @@ def build_artifact_manifest(
return artifacts
def build_submission_manifest(
run_dir: Path,
run_metadata: dict[str, Any],
plan: dict[str, Any],
evidence: list[dict[str, Any]],
assessment_package: dict[str, Any],
) -> dict[str, Any]:
"""Build a portable manifest for the files that make up a review package."""
source_lock = plan["source_lock"]
manifest = {
"id": f"submission-package:{run_metadata['id']}",
"schema_version": "guide-board.submission-package.v1",
"run_id": run_metadata["id"],
"created_at": datetime.now(timezone.utc).isoformat(),
"package_identity": {
"target_profile_ref": run_metadata["target_profile_ref"],
"assessment_profile_ref": run_metadata["assessment_profile_ref"],
"framework_refs": source_lock.get("framework_refs", []),
"extension_refs": source_lock.get("extension_refs", []),
},
"source_lock_ref": "sources.lock.json",
"source_lock": {
"id": source_lock.get("id"),
"schema_version": source_lock.get("schema_version"),
"checksum": _file_entry(run_dir, "sources.lock.json").get("checksum"),
"framework_refs": source_lock.get("framework_refs", []),
"extension_refs": source_lock.get("extension_refs", []),
},
"reports": _existing_file_entries(
run_dir,
[
("assessment-package", "reports/assessment-package.json"),
("markdown-report", "reports/report.md"),
],
),
"normalized_outputs": _existing_file_entries(
run_dir,
[
("evidence", "normalized/evidence.json"),
("findings", "normalized/findings.json"),
("mappings", "normalized/mappings.json"),
],
),
"profile_snapshots": _existing_file_entries(
run_dir,
[
("target-profile", "target-profile.snapshot.json"),
("assessment-profile", "assessment-profile.snapshot.json"),
],
),
"artifact_manifest": assessment_package.get("artifact_manifest", []),
"reported_metadata": _reported_metadata(evidence),
"certification_boundary": assessment_package["certification_boundary"],
}
assert_valid(manifest, "submission-package")
return manifest
def _existing_file_entries(run_dir: Path, refs: list[tuple[str, str]]) -> list[dict[str, Any]]:
entries = []
for entry_id, ref in refs:
entry = _file_entry(run_dir, ref)
if entry:
entry["id"] = entry_id
entries.append(entry)
return entries
def _file_entry(run_dir: Path, ref: str) -> dict[str, Any]:
path = (run_dir / ref).resolve()
try:
path.relative_to(run_dir.resolve())
except ValueError:
return {}
if not path.is_file():
return {}
return {
"path": ref,
"media_type": _media_type(path),
"checksum": f"sha256:{_sha256(path)}",
"size_bytes": path.stat().st_size,
}
def _reported_metadata(evidence: list[dict[str, Any]]) -> list[dict[str, Any]]:
records = []
for item in evidence:
metadata = item.get("facts", {}).get("source_metadata")
if not isinstance(metadata, dict) or not metadata:
continue
records.append(
{
"evidence_ref": item["id"],
"check_id": item["check_id"],
"extension_id": item["extension_id"],
"metadata": metadata,
}
)
return records
def _sha256(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:

116
src/guide_board/execution.py
View File

@@ -7,8 +7,8 @@ from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from guide_board.artifacts import build_artifact_manifest
from guide_board.io import write_json
from guide_board.artifacts import build_artifact_manifest, build_submission_manifest
from guide_board.io import load_json, write_json
from guide_board.mapping import build_mapping_records, summarize_mappings
from guide_board.normalizers import normalize_step_result
from guide_board.planning import build_run_plan
@@ -83,6 +83,7 @@ def run_assessment(
"run_dir": str(run_dir),
"assessment_package": str(run_dir / "reports" / "assessment-package.json"),
"report": str(run_dir / "reports" / "report.md"),
"submission_package": str(run_dir / "reports" / "submission-package.json"),
"retention_summary": str(run_dir / "retention-summary.json"),
}
@@ -155,6 +156,14 @@ def _evidence_for_step(
runner_ref = step.get("runner_ref")
runner_result = run_step(root, run_dir, run_id, plan, step)
runner_result = normalize_step_result(root, run_dir, run_id, plan, step, runner_result)
facts = {
"step_kind": step["kind"],
"runner_ref": runner_ref,
**runner_result["facts"],
}
source_metadata = _source_metadata_for_step(root, plan, step, runner_result)
if source_metadata:
facts["source_metadata"] = source_metadata
return {
"id": f"evidence:{step['id']}",
@@ -164,11 +173,7 @@ def _evidence_for_step(
"subject_ref": plan["target_profile_snapshot"]["id"],
"result": runner_result["result"],
"observations": runner_result["observations"],
"facts": {
"step_kind": step["kind"],
"runner_ref": runner_ref,
**runner_result["facts"],
},
"facts": facts,
"requirement_refs": _requirement_refs(plan, step, runner_result),
"artifact_refs": runner_result["artifact_refs"],
"started_at": now,
@@ -198,6 +203,95 @@ def _runner_requirement_refs(runner_result: dict[str, Any] | None) -> list[str]:
return [ref for ref in refs if isinstance(ref, str)]
def _source_metadata_for_step(
root: Path,
plan: dict[str, Any],
step: dict[str, Any],
runner_result: dict[str, Any],
) -> dict[str, Any]:
runner_ref = step.get("runner_ref")
extension = _extension_snapshot(plan, step["extension_id"])
extension_path = _snapshot_path(root, extension)
manifest = load_json(extension_path / "extension.json")
metadata: dict[str, Any] = {
"extension": {
"id": step["extension_id"],
"version": extension.get("version"),
"metadata": _object_or_empty(manifest.get("metadata")),
}
}
if runner_ref:
entrypoint = _runner_entrypoint(manifest, runner_ref)
metadata["runner"] = {
"id": runner_ref,
"kind": entrypoint.get("kind"),
"metadata": _object_or_empty(entrypoint.get("metadata")),
}
applied_normalizers = runner_result.get("facts", {}).get("normalizer_refs", [])
normalizers = []
if isinstance(applied_normalizers, list):
normalizer_ids = {item for item in applied_normalizers if isinstance(item, str)}
for normalizer in manifest.get("normalizers", []):
if isinstance(normalizer, dict) and normalizer.get("id") in normalizer_ids:
normalizers.append(
{
"id": normalizer["id"],
"kind": normalizer.get("kind"),
"runner_ref": normalizer.get("runner_ref"),
"metadata": _object_or_empty(normalizer.get("metadata")),
}
)
if normalizers:
metadata["normalizers"] = normalizers
reported = _object_or_empty(runner_result.get("metadata"))
if reported:
metadata["reported"] = reported
return _drop_empty_metadata(metadata)
def _extension_snapshot(plan: dict[str, Any], extension_id: str) -> dict[str, Any]:
for extension in plan["extension_snapshots"]:
if extension["id"] == extension_id:
return extension
return {}
def _snapshot_path(root: Path, extension: dict[str, Any]) -> Path:
path = Path(extension["path"])
return path if path.is_absolute() else root / path
def _runner_entrypoint(manifest: dict[str, Any], runner_ref: str) -> dict[str, Any]:
for entrypoint in manifest.get("runner_entrypoints", []):
if entrypoint.get("id") == runner_ref:
return entrypoint
return {}
def _object_or_empty(value: Any) -> dict[str, Any]:
return value if isinstance(value, dict) else {}
def _drop_empty_metadata(value: dict[str, Any]) -> dict[str, Any]:
compact = {}
for key, child in value.items():
if isinstance(child, dict):
child = _drop_empty_metadata(child)
if isinstance(child, list):
child = [
_drop_empty_metadata(item) if isinstance(item, dict) else item
for item in child
]
child = [item for item in child if item]
if child:
compact[key] = child
return compact
def _dedupe(values: list[str]) -> list[str]:
seen = set()
deduped = []
@@ -340,6 +434,14 @@ def _write_run_directory(
_markdown_report(run_metadata, assessment_package),
encoding="utf-8",
)
submission_manifest = build_submission_manifest(
run_dir,
run_metadata,
plan,
evidence,
assessment_package,
)
write_json(run_dir / "reports" / "submission-package.json", submission_manifest)
def _markdown_report(run_metadata: dict[str, Any], package: dict[str, Any]) -> str:

12
src/guide_board/normalizers.py
View File

@@ -127,6 +127,7 @@ def _run_normalizer(
},
"artifact_refs": runner_result.get("artifact_refs", []),
"requirement_refs": runner_result.get("requirement_refs", []),
"metadata": runner_result.get("metadata", {}),
}
if not isinstance(result, dict):
@@ -160,6 +161,12 @@ def _merge_result(
_string_list(base.get("requirement_refs", []))
+ _string_list(update.get("requirement_refs", []))
)
if "metadata" in update:
metadata = dict(base.get("metadata", {}))
update_metadata = update.get("metadata", {})
if isinstance(update_metadata, dict):
metadata.update(update_metadata)
merged["metadata"] = metadata
return _coerce_result(merged)
@@ -173,6 +180,7 @@ def _coerce_result(value: dict[str, Any]) -> dict[str, Any]:
"facts": facts,
"artifact_refs": _string_list(value.get("artifact_refs", [])),
"requirement_refs": _string_list(value.get("requirement_refs", [])),
"metadata": _object_or_empty(value.get("metadata")),
}
@@ -189,6 +197,10 @@ def _string_list(value: Any) -> list[str]:
return [item for item in value if isinstance(item, str)]
def _object_or_empty(value: Any) -> dict[str, Any]:
return value if isinstance(value, dict) else {}
def _dedupe(values: list[str]) -> list[str]:
seen = set()
deduped = []

284
src/guide_board/planning.py
View File

@@ -2,6 +2,7 @@
from __future__ import annotations
import hashlib
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
@@ -96,6 +97,16 @@ def build_run_plan(
}
)
source_lock = _build_source_lock(
root,
target_path,
assessment_path,
target,
assessment,
[extensions[extension_id] for extension_id in selected_extensions],
)
assert_valid(source_lock, "source-lock")
plan = {
"id": f"plan-{_timestamp()}",
"assessment_profile_snapshot": assessment,
@@ -109,10 +120,7 @@ def build_run_plan(
}
for extension_id in selected_extensions
],
"source_lock": {
"framework_refs": assessment["framework_refs"],
"extension_refs": selected_extensions,
},
"source_lock": source_lock,
"profile_paths": {
"target_profile_path": str(target_path.resolve()),
"assessment_profile_path": str(assessment_path.resolve()),
@@ -208,6 +216,270 @@ def _load_extension_profile_schema(
return load_json(schema_path)
def _build_source_lock(
root: Path,
target_path: Path,
assessment_path: Path,
target: dict[str, Any],
assessment: dict[str, Any],
extensions: list[Extension],
) -> dict[str, Any]:
framework_refs = assessment["framework_refs"]
extension_refs = [extension.id for extension in extensions]
return {
"id": f"source-lock:{assessment['id']}:{target['id']}",
"schema_version": "guide-board.source-lock.v1",
"created_at": _now(),
"framework_refs": framework_refs,
"extension_refs": extension_refs,
"frameworks": _framework_records(framework_refs, extensions),
"extensions": [_extension_source_record(root, extension) for extension in extensions],
"mapping_sets": _mapping_source_records(root, extensions),
"profiles": {
"target": _file_source_record(
"target-profile",
target["id"],
target_path,
"target-profile.snapshot.json",
),
"assessment": _file_source_record(
"assessment-profile",
assessment["id"],
assessment_path,
"assessment-profile.snapshot.json",
),
},
"policy_refs": {
"expectations": _optional_policy_source_record(
root,
assessment_path,
assessment.get("expectations_ref"),
"expectation-set",
),
"waivers": _optional_policy_source_record(
root,
assessment_path,
assessment.get("waivers_ref"),
"waiver-set",
),
},
"authorities": _authority_source_records(extensions),
"metadata_hooks": {
"runner_entrypoints": _entrypoint_metadata_records(extensions),
"normalizers": _normalizer_metadata_records(extensions),
},
}
def _framework_records(
framework_refs: list[str],
extensions: list[Extension],
) -> list[dict[str, Any]]:
records = []
for framework_ref in framework_refs:
declaring_extensions = [
extension.id
for extension in extensions
if framework_ref in _manifest_framework_ids(extension.manifest)
]
records.append(
{
"id": framework_ref,
"version": _version_hint(framework_ref),
"declared_by_extensions": declaring_extensions,
}
)
return records
def _extension_source_record(root: Path, extension: Extension) -> dict[str, Any]:
manifest_path = extension.path / "extension.json"
return {
"id": extension.id,
"version": extension.manifest["version"],
"path": _extension_path_ref(root, extension.path),
"source": extension.source,
"manifest_path": _display_path(root, manifest_path),
"manifest_checksum": _checksum_if_file(manifest_path),
"supported_frameworks": _manifest_framework_ids(extension.manifest),
"authorities": _authority_ids(extension.manifest.get("authorities", [])),
"certification_boundary": extension.manifest["certification_boundary"],
"metadata": _object_or_empty(extension.manifest.get("metadata")),
}
def _mapping_source_records(root: Path, extensions: list[Extension]) -> list[dict[str, Any]]:
records = []
for extension in extensions:
for mapping_id in extension.manifest.get("mappings", []):
if not isinstance(mapping_id, str):
continue
mapping_path = extension.path / "mappings" / f"{mapping_id}.json"
record = {
"id": mapping_id,
"extension_id": extension.id,
"path": _display_path(root, mapping_path),
"exists": mapping_path.is_file(),
"checksum": _checksum_if_file(mapping_path),
"framework_refs": [],
}
if mapping_path.is_file():
mapping_set = load_json(mapping_path)
record["framework_refs"] = _string_list(mapping_set.get("framework_refs", []))
records.append(record)
return records
def _file_source_record(
kind: str,
profile_id: str,
path: Path,
snapshot_ref: str,
) -> dict[str, Any]:
return {
"kind": kind,
"id": profile_id,
"path": str(path.resolve()),
"checksum": _checksum_if_file(path),
"snapshot_ref": snapshot_ref,
}
def _optional_policy_source_record(
root: Path,
assessment_path: Path,
ref: Any,
kind: str,
) -> dict[str, Any] | None:
if not isinstance(ref, str) or not ref:
return None
path = _resolve_assessment_ref(root, assessment_path, ref)
return {
"kind": kind,
"ref": ref,
"path": str(path.resolve()),
"exists": path.is_file(),
"checksum": _checksum_if_file(path),
}
def _authority_source_records(extensions: list[Extension]) -> list[dict[str, Any]]:
records = []
for extension in extensions:
for authority in extension.manifest.get("authorities", []):
if isinstance(authority, str):
records.append({"id": authority, "extension_id": extension.id})
elif isinstance(authority, dict):
record = {
"id": authority.get("id"),
"extension_id": extension.id,
}
for key in ("name", "version", "source_url", "license", "access"):
if key in authority:
record[key] = authority[key]
records.append(record)
return [record for record in records if isinstance(record.get("id"), str)]
def _entrypoint_metadata_records(extensions: list[Extension]) -> list[dict[str, Any]]:
records = []
for extension in extensions:
for entrypoint in extension.manifest.get("runner_entrypoints", []):
if not isinstance(entrypoint, dict):
continue
records.append(
{
"extension_id": extension.id,
"id": entrypoint.get("id"),
"kind": entrypoint.get("kind"),
"metadata": _object_or_empty(entrypoint.get("metadata")),
}
)
return records
def _normalizer_metadata_records(extensions: list[Extension]) -> list[dict[str, Any]]:
records = []
for extension in extensions:
for normalizer in extension.manifest.get("normalizers", []):
if not isinstance(normalizer, dict):
continue
records.append(
{
"extension_id": extension.id,
"id": normalizer.get("id"),
"kind": normalizer.get("kind"),
"runner_ref": normalizer.get("runner_ref"),
"metadata": _object_or_empty(normalizer.get("metadata")),
}
)
return records
def _manifest_framework_ids(manifest: dict[str, Any]) -> list[str]:
values = []
for framework in manifest.get("supported_frameworks", []):
if isinstance(framework, str):
values.append(framework)
elif isinstance(framework, dict) and isinstance(framework.get("id"), str):
values.append(framework["id"])
return values
def _authority_ids(authorities: list[Any]) -> list[str]:
values = []
for authority in authorities:
if isinstance(authority, str):
values.append(authority)
elif isinstance(authority, dict) and isinstance(authority.get("id"), str):
values.append(authority["id"])
return values
def _resolve_assessment_ref(root: Path, assessment_path: Path, ref: str) -> Path:
ref_path = Path(ref)
if ref_path.is_absolute():
return ref_path
root_relative = root / ref_path
if root_relative.exists():
return root_relative
return assessment_path.resolve().parent / ref_path
def _checksum_if_file(path: Path) -> str | None:
if not path.is_file():
return None
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return f"sha256:{digest.hexdigest()}"
def _version_hint(ref: str) -> str | None:
for part in reversed(ref.replace("-", ".").split(".")):
if len(part) > 1 and part[0].lower() == "v" and any(char.isdigit() for char in part[1:]):
return part
return None
def _object_or_empty(value: Any) -> dict[str, Any]:
return value if isinstance(value, dict) else {}
def _string_list(value: Any) -> list[str]:
if not isinstance(value, list):
return []
return [item for item in value if isinstance(item, str)]
def _display_path(root: Path, path: Path) -> str:
try:
return str(path.resolve().relative_to(root.resolve()))
except ValueError:
return str(path.resolve())
def _extension_path_ref(root: Path, path: Path) -> str:
try:
return str(path.resolve().relative_to(root.resolve()))
@@ -215,5 +487,9 @@ def _extension_path_ref(root: Path, path: Path) -> str:
return str(path.resolve())
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _timestamp() -> str:
return datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")

1
src/guide_board/retention.py
View File

@@ -45,6 +45,7 @@ def build_retention_summary(
"report_refs": [
"reports/assessment-package.json",
"reports/report.md",
"reports/submission-package.json",
],
"artifact_retention": {
"policy": plan["assessment_profile_snapshot"].get("retention_policy", {}),

31
src/guide_board/runners.py
View File

@@ -45,6 +45,8 @@ def run_step(
"runner_kind": "external",
},
"artifact_refs": [],
"requirement_refs": [],
"metadata": _object_or_empty(entrypoint.get("metadata")),
}
if entrypoint["kind"] == "command":
return _run_command(root, run_dir, run_id, plan, step, extension_path, entrypoint)
@@ -63,6 +65,8 @@ def _no_runner_result(step: dict[str, Any]) -> dict[str, Any]:
"runner_kind": None,
},
"artifact_refs": [],
"requirement_refs": [],
"metadata": {},
}
@@ -118,6 +122,8 @@ def _run_python_module(
"error_type": type(exc).__name__,
},
"artifact_refs": [],
"requirement_refs": [],
"metadata": _object_or_empty(entrypoint.get("metadata")),
}
if not isinstance(result, dict):
raise ValidationError(f"{entrypoint['id']}: runner must return an object")
@@ -126,6 +132,8 @@ def _run_python_module(
"observations": result.get("observations", []),
"facts": result.get("facts", {}),
"artifact_refs": result.get("artifact_refs", []),
"requirement_refs": result.get("requirement_refs", []),
"metadata": _merge_metadata(entrypoint.get("metadata"), result.get("metadata")),
}
@@ -192,6 +200,8 @@ def _run_command(
"command": command,
},
"artifact_refs": [str(context_path.relative_to(run_dir))],
"requirement_refs": [],
"metadata": _object_or_empty(entrypoint.get("metadata")),
}
except subprocess.TimeoutExpired:
return {
@@ -206,6 +216,8 @@ def _run_command(
"command": command,
},
"artifact_refs": [str(context_path.relative_to(run_dir))],
"requirement_refs": [],
"metadata": _object_or_empty(entrypoint.get("metadata")),
}
parsed = _parse_runner_stdout(completed.stdout)
@@ -225,6 +237,8 @@ def _run_command(
"command": command,
},
"artifact_refs": [str(context_path.relative_to(run_dir))],
"requirement_refs": [],
"metadata": _object_or_empty(entrypoint.get("metadata")),
}
facts = parsed.get("facts", {})
@@ -245,6 +259,9 @@ def _run_command(
if not isinstance(artifact_refs, list):
artifact_refs = []
artifact_refs.append(str(context_path.relative_to(run_dir)))
requirement_refs = parsed.get("requirement_refs", [])
if not isinstance(requirement_refs, list):
requirement_refs = []
result = parsed.get("result", "unknown")
if completed.returncode != 0 and result in {"pass", "warning", "manual", "skipped"}:
@@ -258,6 +275,8 @@ def _run_command(
"observations": observations,
"facts": facts,
"artifact_refs": artifact_refs,
"requirement_refs": requirement_refs,
"metadata": _merge_metadata(entrypoint.get("metadata"), parsed.get("metadata")),
}
@@ -328,5 +347,17 @@ def _parse_runner_stdout(stdout: str) -> dict[str, Any] | None:
return parsed
def _merge_metadata(*values: Any) -> dict[str, Any]:
merged: dict[str, Any] = {}
for value in values:
if isinstance(value, dict):
merged.update(value)
return merged
def _object_or_empty(value: Any) -> dict[str, Any]:
return value if isinstance(value, dict) else {}
def _safe_id(value: str) -> str:
return "".join(char if char.isalnum() or char in {"-", "_"} else "_" for char in value)

55
tests/test_core.py
View File

@@ -75,6 +75,17 @@ class CoreArchitectureTests(unittest.TestCase):
plan["ordered_steps"][1]["requirement_refs"],
["guide-board.sample-readiness.v0.profile-shape"],
)
assert_valid(plan["source_lock"], "source-lock")
self.assertEqual(plan["source_lock"]["schema_version"], "guide-board.source-lock.v1")
self.assertEqual(plan["source_lock"]["framework_refs"], ["guide-board.sample-readiness.v0"])
self.assertEqual(plan["source_lock"]["extension_refs"], ["sample-noop"])
self.assertEqual(
plan["source_lock"]["profiles"]["target"]["snapshot_ref"],
"target-profile.snapshot.json",
)
self.assertTrue(plan["source_lock"]["profiles"]["target"]["checksum"].startswith("sha256:"))
self.assertEqual(plan["source_lock"]["mapping_sets"][0]["id"], "sample-readiness-map")
self.assertTrue(plan["source_lock"]["mapping_sets"][0]["checksum"].startswith("sha256:"))
def test_runs_external_extension_from_separate_repo(self) -> None:
with TemporaryDirectory() as temporary_directory:
@@ -237,8 +248,37 @@ class CoreArchitectureTests(unittest.TestCase):
check_evidence["artifact_refs"],
["artifacts/sdk-fixture/native-result.json"],
)
self.assertEqual(
check_evidence["facts"]["source_metadata"]["runner"]["metadata"]["harness_version"],
"1.0.0",
)
self.assertEqual(
check_evidence["facts"]["source_metadata"]["reported"]["native_result_id"],
"sdk-fixture-native-result",
)
self.assertEqual(mappings[0]["target_id"], "normalizer-plugin")
self.assertEqual(assessment_package["summary"], {"pass": 1, "skipped": 1})
self.assertEqual(
assessment_package["source_lock"]["metadata_hooks"]["runner_entrypoints"][0][
"metadata"
]["harness_id"],
"sdk-fixture-native-probe",
)
submission_package = json.loads(
(run_dir / "reports" / "submission-package.json").read_text(encoding="utf-8")
)
assert_valid(submission_package, "submission-package")
self.assertEqual(submission_package["source_lock"]["id"], "source-lock:sdk-fixture-assessment:sdk-fixture-target")
self.assertEqual(
submission_package["reported_metadata"][1]["metadata"]["reported"][
"native_result_id"
],
"sdk-fixture-native-result",
)
self.assertEqual(
submission_package["artifact_manifest"][0]["checksum"],
assessment_package["artifact_manifest"][0]["checksum"],
)
def test_runs_sample_noop_assessment(self) -> None:
with TemporaryDirectory() as temporary_directory:
@@ -256,6 +296,7 @@ class CoreArchitectureTests(unittest.TestCase):
self.assertTrue((run_dir / "normalized" / "evidence.json").exists())
self.assertTrue((run_dir / "reports" / "assessment-package.json").exists())
self.assertTrue((run_dir / "reports" / "report.md").exists())
self.assertTrue((run_dir / "reports" / "submission-package.json").exists())
retention = json.loads(
(run_dir / "retention-summary.json").read_text(encoding="utf-8")
)
@@ -263,12 +304,26 @@ class CoreArchitectureTests(unittest.TestCase):
result["retention_summary"],
str(run_dir / "retention-summary.json"),
)
self.assertEqual(
result["submission_package"],
str(run_dir / "reports" / "submission-package.json"),
)
self.assertEqual(retention["summary"]["status"], "completed")
self.assertEqual(retention["summary"]["artifact_count"], 0)
self.assertIn("reports/submission-package.json", retention["report_refs"])
self.assertEqual(
retention["artifact_retention"]["policy"],
{"raw_artifact_days": 0, "summary_days": 365},
)
submission = json.loads(
(run_dir / "reports" / "submission-package.json").read_text(encoding="utf-8")
)
assert_valid(submission, "submission-package")
self.assertEqual(submission["package_identity"]["target_profile_ref"], "sample-repository")
self.assertEqual(
[entry["path"] for entry in submission["reports"]],
["reports/assessment-package.json", "reports/report.md"],
)
self.assertEqual(
[run["run_id"] for run in list_retained_runs(Path(temporary_directory))],
[result["run_id"]],

49
workplans/GUIDE-BOARD-WP-0004-source-lock-and-submission-package-baseline.md
View File

@@ -4,12 +4,12 @@ type: workplan
title: "Source Lock And Submission Package Baseline"
repo: guide-board
domain: markitect
status: active
status: completed
owner: codex
planning_priority: high
planning_order: 4
created: "2026-05-15"
updated: "2026-05-15"
updated: "2026-05-16"
state_hub_workstream_id: "6dd2832b-d1d9-43bc-ad5c-d16f399930dc"
---
@@ -41,7 +41,7 @@ submission rules, and licensed or restricted assets remain extension-owned.
```task
id: GUIDE-BOARD-WP-0004-T001
status: todo
status: done
priority: high
state_hub_task_id: "d5a7a18f-941b-47b8-9992-2cb54bc5ad06"
```
@@ -55,11 +55,20 @@ Acceptance:
- Keep the schema backward-compatible with existing retained runs.
- Add tests for source lock shape and retained run compatibility.
Progress:
- Added `docs/schemas/source-lock.schema.json`.
- Expanded run-plan source locks with framework, extension, mapping-set,
profile snapshot, policy-ref, authority, and metadata-hook records.
- Preserved the original `framework_refs` and `extension_refs` fields for
retained-run compatibility.
- Added tests for source-lock shape and older retained summary compatibility.
## D4.2 - Harness And Extension Metadata Hooks
```task
id: GUIDE-BOARD-WP-0004-T002
status: todo
status: done
priority: high
state_hub_task_id: "7abd5a66-5784-41b9-a361-6572290923cc"
```
@@ -75,11 +84,20 @@ Acceptance:
provide this metadata yet.
- Cover the SDK fixture and at least one no-metadata extension path in tests.
Progress:
- Added optional manifest metadata for extensions, authorities, frameworks,
runner entrypoints, and normalizers.
- Preserved runner and normalizer returned `metadata` and requirement refs.
- Recorded merged metadata under evidence `facts.source_metadata`.
- Updated `sdk-fixture` to exercise harness, test-suite, adapter, source URL,
and native result metadata while keeping `sample-noop` as a no-metadata path.
## D4.3 - Submission Package Manifest
```task
id: GUIDE-BOARD-WP-0004-T003
status: todo
status: done
priority: medium
state_hub_task_id: "c54273d6-1fc2-4444-92cf-74f2a5e614ec"
```
@@ -95,11 +113,21 @@ Acceptance:
packs.
- Document how this differs from an authority-specific final submission.
Progress:
- Added `docs/schemas/submission-package.schema.json`.
- Wrote `reports/submission-package.json` for each run.
- Included package identity, source lock checksum, report paths, normalized
output paths, profile snapshots, artifact manifest entries, reported
metadata, and the certification boundary.
- Exposed the submission manifest path in CLI/service run results and retained
report refs.
## D4.4 - Documentation And Acceptance Tests
```task
id: GUIDE-BOARD-WP-0004-T004
status: todo
status: done
priority: medium
state_hub_task_id: "ad37baeb-973c-4399-96d0-c9cb7fc6b761"
```
@@ -113,6 +141,15 @@ Acceptance:
- Include compatibility notes for older retained runs.
- Keep the output paths aligned with existing CLI and service result retrieval.
Progress:
- Updated assessment operations, extension SDK, architecture blueprint, and
README references.
- Added focused unit assertions for the sample and SDK fixture assessment
outputs.
- Kept retained run listing compatible with older `retention-summary.json`
files that do not reference a submission manifest.
## Definition Of Done
- Every new run writes a richer source lock and submission package manifest.