Complete extension SDK maturity

This commit is contained in:
2026-05-15 15:34:55 +02:00
parent 67f2fc5346
commit 6758b3992c
19 changed files with 680 additions and 14 deletions

View File

@@ -1,8 +1,8 @@
<!-- custodian-brief: generated by fix-consistency — do not edit manually --> <!-- custodian-brief: generated by fix-consistency — do not edit manually -->
# Custodian Brief — guide-board # Custodian Brief — guide-board
**Domain:** markitect **Domain:** markitect
**Last synced:** 2026-05-15 13:30 UTC **Last synced:** 2026-05-15 13:30 UTC
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)* **State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
## Active Workstreams ## Active Workstreams

View File

@@ -44,7 +44,9 @@ those contracts for service and container operation; see
[docs/LOCAL-SERVICE-API.md](docs/LOCAL-SERVICE-API.md). [docs/LOCAL-SERVICE-API.md](docs/LOCAL-SERVICE-API.md).
The `sample-noop` extension exercises the guide-board contracts without invoking The `sample-noop` extension exercises the guide-board contracts without invoking
an external harness. `open-cmis-tck` is the first real seed extension. an external harness. `sdk-fixture` demonstrates the extension SDK contracts for
schemas, normalizers, mappings, and fixture profiles. `open-cmis-tck` is the
first real seed extension.
See: See:
@@ -60,3 +62,5 @@ See:
- [docs/SERVICE-JOB-DURABILITY.md](docs/SERVICE-JOB-DURABILITY.md) - [docs/SERVICE-JOB-DURABILITY.md](docs/SERVICE-JOB-DURABILITY.md)
- [extensions/CANDIDATES.md](extensions/CANDIDATES.md) - [extensions/CANDIDATES.md](extensions/CANDIDATES.md)
- [workplans/GUIDE-BOARD-WP-0001-bootstrapping.md](workplans/GUIDE-BOARD-WP-0001-bootstrapping.md) - [workplans/GUIDE-BOARD-WP-0001-bootstrapping.md](workplans/GUIDE-BOARD-WP-0001-bootstrapping.md)
- [workplans/GUIDE-BOARD-WP-0002-assessment-operations-baseline.md](workplans/GUIDE-BOARD-WP-0002-assessment-operations-baseline.md)
- [workplans/GUIDE-BOARD-WP-0003-extension-sdk-maturity.md](workplans/GUIDE-BOARD-WP-0003-extension-sdk-maturity.md)

View File

@@ -65,6 +65,8 @@ when a wrapper script or container entrypoint should keep commands shorter.
For the repeatable external extension acceptance path, including validation, For the repeatable external extension acceptance path, including validation,
planning, live execution, and retained result review, see planning, live execution, and retained result review, see
`docs/EXTERNAL-EXTENSION-ACCEPTANCE.md`. `docs/EXTERNAL-EXTENSION-ACCEPTANCE.md`.
For extension-author contracts such as profile schema descriptors and
normalizer plug-ins, see `docs/EXTENSION-SDK.md`.
## CLI Results ## CLI Results

View File

@@ -75,6 +75,8 @@ The key runtime fields are:
- `check_groups`: named groups that assessment profiles can select. - `check_groups`: named groups that assessment profiles can select.
- `preflight_runner`: optional runner ID used before selected check groups. - `preflight_runner`: optional runner ID used before selected check groups.
- `runner_entrypoints`: concrete runner declarations. - `runner_entrypoints`: concrete runner declarations.
- `normalizers`: optional plug-ins that convert native runner output into the
stable runner-result shape before evidence is written.
- `mappings`: mapping set IDs under `mappings/<mapping-id>.json`. - `mappings`: mapping set IDs under `mappings/<mapping-id>.json`.
- `certification_boundary`: explicit statement of what the extension does not - `certification_boundary`: explicit statement of what the extension does not
certify. certify.
@@ -283,6 +285,71 @@ or `infrastructure_error`, downstream check groups for that extension are not
executed; they receive `blocked` evidence with `blocked_reason: executed; they receive `blocked` evidence with `blocked_reason:
preflight_failed`. preflight_failed`.
## Normalizer Plug-ins
Runners can keep returning guide-board-ready result objects directly. When a
runner wraps a native harness or scanner that writes its own result format, the
extension can add a normalizer descriptor:
```json
{
"id": "native-probe-normalizer",
"kind": "python_module",
"module_path": "normalizers/native_probe.py",
"callable": "normalize",
"runner_ref": "native-probe",
"description": "Converts native runner output into guide-board evidence."
}
```
Normalizers are declared in `extension.json` under `normalizers`. The original
string shorthand remains valid for descriptive-only entries, but only descriptor
objects are loaded and invoked by the core.
The first supported normalizer kind is `python_module`. Its module path is
resolved relative to the extension root and must stay inside that root. The
callable receives one context object:
- `root`: guide-board core root path as a string.
- `extension_path`: extension root path as a string.
- `run_dir`: output run directory path as a string.
- `run_id`: current run ID.
- `plan`: full run plan snapshot.
- `step`: the step being normalized.
- `target_profile`: target profile snapshot.
- `assessment_profile`: assessment profile snapshot.
- `normalizer`: manifest normalizer descriptor.
- `runner_result`: the current runner-result object.
A normalizer returns any subset of the runner-result fields:
```python
def normalize(context: dict) -> dict:
return {
"result": "pass",
"observations": ["Native result was normalized."],
"facts": {"native_status": "ok"},
"artifact_refs": ["artifacts/native-result.json"],
"requirement_refs": ["framework.requirement"],
}
```
The core merges the normalizer output over the runner result:
- `result` replaces the previous result.
- `observations` are appended.
- `facts` are merged.
- `artifact_refs` and `requirement_refs` are deduplicated.
- `normalizer_refs` is recorded in evidence facts when any normalizer runs.
If a normalizer raises an exception, the step becomes
`infrastructure_error` evidence and the run still produces its normal artifact
set.
The bundled `extensions/sdk-fixture` extension is the copyable reference path
for profile schemas, a native-output runner, a normalizer, mappings, and fixture
profiles.
## Result Statuses ## Result Statuses
Initial statuses: Initial statuses:
@@ -303,11 +370,14 @@ Initial statuses:
## Current Extension Examples ## Current Extension Examples
- `sample-noop`: no runner, used to validate the core contracts. - `sample-noop`: no runner, used to validate the core contracts.
- `sdk-fixture`: compact SDK fixture covering profile schemas, runner output,
normalizer invocation, mapping, and fixture profiles.
- `open-cmis-tck`: provides a Python CMIS Browser Binding preflight runner and - `open-cmis-tck`: provides a Python CMIS Browser Binding preflight runner and
declares the future external OpenCMIS TCK runner. declares the future external OpenCMIS TCK runner.
## Next SDK Steps ## Next SDK Steps
- Add normalizer plug-in contracts. - Broaden normalizer examples as real external extensions adopt native harness
- Add extension-owned schema validation for domain-specific target profile result formats.
fields. - Add more extension-owned schema validation examples for assessment-specific
domain constraints.

View File

@@ -15,6 +15,9 @@ domain-specific harness logic into the core.
runtime dependencies, and harness behavior remain owned by that extension runtime dependencies, and harness behavior remain owned by that extension
repository. repository.
For a dependency-light SDK reference extension that can be copied into a
temporary external repository, see `extensions/sdk-fixture`.
## Acceptance Stages ## Acceptance Stages
Run these stages from the guide-board repository. Run these stages from the guide-board repository.

View File

@@ -93,7 +93,22 @@
} }
} }
}, },
"normalizers": { "type": "array", "items": { "type": "string" } }, "normalizers": {
"type": "array",
"items": {
"type": ["string", "object"],
"additionalProperties": false,
"required": ["id", "kind", "module_path", "callable"],
"properties": {
"id": { "type": "string" },
"kind": { "type": "string", "enum": ["python_module"] },
"module_path": { "type": "string" },
"callable": { "type": "string" },
"runner_ref": { "type": ["string", "null"] },
"description": { "type": ["string", "null"] }
}
}
},
"mappings": { "type": "array", "items": { "type": "string" } }, "mappings": { "type": "array", "items": { "type": "string" } },
"report_fragments": { "type": "array", "items": { "type": "string" } }, "report_fragments": { "type": "array", "items": { "type": "string" } },
"dependencies": { "type": "array", "items": { "type": "string" } }, "dependencies": { "type": "array", "items": { "type": "string" } },

View File

@@ -0,0 +1,13 @@
# SDK Fixture Extension
`sdk-fixture` is a dependency-light guide-board extension used to exercise the
extension SDK contracts. It is intentionally small and is not a real assessment
program.
The fixture demonstrates:
- extension-owned target and assessment profile schemas,
- a Python runner that writes native output,
- a Python normalizer that converts native output into guide-board evidence,
- a mapping set for normalized requirement refs,
- copyable profiles for SDK acceptance tests.

View File

@@ -0,0 +1,67 @@
{
"id": "sdk-fixture",
"name": "SDK Fixture Extension",
"version": "0.1.0",
"extension_type": "repository_quality",
"lifecycle_status": "incubating",
"supported_frameworks": [
"guide-board.sdk-fixture.v1"
],
"authorities": [],
"profile_schemas": [
"target-profile",
"assessment-profile",
{
"id": "sdk-fixture-target",
"profile_kind": "target",
"path": "schemas/sdk-fixture-target.schema.json",
"subject_type": "sdk-fixture-target",
"description": "Requires the target shape used by the SDK fixture runner."
},
{
"id": "sdk-fixture-assessment",
"profile_kind": "assessment",
"path": "schemas/sdk-fixture-assessment.schema.json",
"description": "Requires the runtime policy used by the SDK fixture normalizer test."
}
],
"check_groups": [
{
"id": "native-output",
"name": "Native Output Normalization",
"check_type": "repository_quality",
"requirement_refs": [
"guide-board.sdk-fixture.v1.native-output"
],
"runner_ref": "native-probe"
}
],
"preflight_runner": null,
"runner_entrypoints": [
{
"id": "native-probe",
"kind": "python_module",
"module_path": "runners/native_probe.py",
"callable": "run",
"command": null,
"description": "Writes a tiny native result artifact for the SDK fixture normalizer."
}
],
"normalizers": [
{
"id": "native-probe-normalizer",
"kind": "python_module",
"module_path": "normalizers/native_probe.py",
"callable": "normalize",
"runner_ref": "native-probe",
"description": "Converts the SDK fixture native result artifact into guide-board evidence."
}
],
"mappings": [
"sdk-fixture-map"
],
"report_fragments": [],
"dependencies": [],
"restricted_assets": [],
"certification_boundary": "SDK fixture only. It does not certify any product, process, or repository."
}

View File

@@ -0,0 +1,16 @@
{
"id": "sdk-fixture-map",
"extension_id": "sdk-fixture",
"framework_refs": [
"guide-board.sdk-fixture.v1"
],
"mappings": [
{
"requirement_ref": "guide-board.sdk-fixture.v1.native-output",
"target_type": "sdk_contract",
"target_id": "normalizer-plugin",
"label": "Normalizer Plug-in Contract",
"description": "The extension runner can emit native output that a normalizer converts into guide-board evidence."
}
]
}

View File

@@ -0,0 +1,28 @@
"""SDK fixture normalizer for native runner output."""
from __future__ import annotations
import json
from pathlib import Path
def normalize(context: dict) -> dict:
run_dir = Path(context["run_dir"])
runner_result = context["runner_result"]
artifact_ref = runner_result["facts"]["native_result_ref"]
native_result = json.loads((run_dir / artifact_ref).read_text(encoding="utf-8"))
native_status = native_result.get("native_status")
result = "pass" if native_status == "ok" else "fail"
return {
"result": result,
"observations": native_result.get("observations", []),
"facts": {
"native_status": native_status,
"native_score": native_result.get("native_score"),
"normalized_by": "native-probe-normalizer"
},
"artifact_refs": [
artifact_ref
],
"requirement_refs": native_result.get("requirement_refs", []),
}

View File

@@ -0,0 +1,33 @@
{
"id": "sdk-fixture-assessment",
"framework_refs": [
"guide-board.sdk-fixture.v1"
],
"extension_refs": [
"sdk-fixture"
],
"target_profile_ref": "sdk-fixture-target",
"selected_check_groups": {
"sdk-fixture": [
"native-output"
]
},
"expectations_ref": null,
"waivers_ref": null,
"output_policy": {
"report_formats": [
"json",
"markdown"
],
"artifact_retention": "raw-logs-plus-summary"
},
"retention_policy": {
"summary_days": 365,
"raw_artifact_days": 30
},
"runtime_policy": {
"offline": true,
"timeout_seconds": 30,
"fixture_mode": "native-result"
}
}

View File

@@ -0,0 +1,18 @@
{
"id": "sdk-fixture-target",
"subject_type": "sdk-fixture-target",
"subject_name": "SDK Fixture Target",
"environment": "test",
"scope": [
"Extension SDK fixture validation"
],
"endpoints": [],
"artifacts": [
"extension.json"
],
"credentials_ref": null,
"declared_capabilities": [
"guide-board.sdk-fixture.v1.native-output"
],
"known_gaps": []
}

View File

@@ -0,0 +1,36 @@
"""SDK fixture runner that writes a native result artifact."""
from __future__ import annotations
import json
from pathlib import Path
def run(context: dict) -> dict:
run_dir = Path(context["run_dir"])
artifact_path = run_dir / "artifacts" / "sdk-fixture" / "native-result.json"
artifact_path.parent.mkdir(parents=True, exist_ok=True)
native_result = {
"native_status": "ok",
"native_score": 98,
"observations": [
"SDK fixture native probe completed."
],
"requirement_refs": [
"guide-board.sdk-fixture.v1.native-output"
],
}
artifact_path.write_text(json.dumps(native_result, indent=2, sort_keys=True), encoding="utf-8")
artifact_ref = str(artifact_path.relative_to(run_dir))
return {
"result": "unknown",
"observations": [
"SDK fixture runner wrote native output for normalization."
],
"facts": {
"native_result_ref": artifact_ref
},
"artifact_refs": [
artifact_ref
],
}

View File

@@ -0,0 +1,19 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "SDK Fixture Assessment Profile",
"type": "object",
"required": [
"runtime_policy"
],
"properties": {
"runtime_policy": {
"type": "object",
"required": [
"fixture_mode"
],
"properties": {
"fixture_mode": { "enum": ["native-result"] }
}
}
}
}

View File

@@ -0,0 +1,15 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "SDK Fixture Target Profile",
"type": "object",
"required": [
"subject_type",
"artifacts",
"declared_capabilities"
],
"properties": {
"subject_type": { "enum": ["sdk-fixture-target"] },
"artifacts": { "type": "array", "minItems": 1 },
"declared_capabilities": { "type": "array", "minItems": 1 }
}
}

View File

@@ -10,6 +10,7 @@ from typing import Any
from guide_board.artifacts import build_artifact_manifest from guide_board.artifacts import build_artifact_manifest
from guide_board.io import write_json from guide_board.io import write_json
from guide_board.mapping import build_mapping_records, summarize_mappings from guide_board.mapping import build_mapping_records, summarize_mappings
from guide_board.normalizers import normalize_step_result
from guide_board.planning import build_run_plan from guide_board.planning import build_run_plan
from guide_board.policy import apply_policy from guide_board.policy import apply_policy
from guide_board.retention import build_retention_summary from guide_board.retention import build_retention_summary
@@ -153,6 +154,7 @@ def _evidence_for_step(
now = _now() now = _now()
runner_ref = step.get("runner_ref") runner_ref = step.get("runner_ref")
runner_result = run_step(root, run_dir, run_id, plan, step) runner_result = run_step(root, run_dir, run_id, plan, step)
runner_result = normalize_step_result(root, run_dir, run_id, plan, step, runner_result)
return { return {
"id": f"evidence:{step['id']}", "id": f"evidence:{step['id']}",
@@ -167,17 +169,44 @@ def _evidence_for_step(
"runner_ref": runner_ref, "runner_ref": runner_ref,
**runner_result["facts"], **runner_result["facts"],
}, },
"requirement_refs": _requirement_refs(plan, step), "requirement_refs": _requirement_refs(plan, step, runner_result),
"artifact_refs": runner_result["artifact_refs"], "artifact_refs": runner_result["artifact_refs"],
"started_at": now, "started_at": now,
"completed_at": now, "completed_at": now,
} }
def _requirement_refs(plan: dict[str, Any], step: dict[str, Any]) -> list[str]: def _requirement_refs(
plan: dict[str, Any],
step: dict[str, Any],
runner_result: dict[str, Any] | None = None,
) -> list[str]:
refs = []
if step["kind"] != "check_group": if step["kind"] != "check_group":
return _runner_requirement_refs(runner_result)
refs.extend(step.get("requirement_refs", []))
refs.extend(_runner_requirement_refs(runner_result))
return _dedupe(refs)
def _runner_requirement_refs(runner_result: dict[str, Any] | None) -> list[str]:
if not runner_result:
return [] return []
return list(step.get("requirement_refs", [])) refs = runner_result.get("requirement_refs", [])
if not isinstance(refs, list):
return []
return [ref for ref in refs if isinstance(ref, str)]
def _dedupe(values: list[str]) -> list[str]:
seen = set()
deduped = []
for value in values:
if value in seen:
continue
seen.add(value)
deduped.append(value)
return deduped
def _findings_for_evidence(run_id: str, evidence: list[dict[str, Any]]) -> list[dict[str, Any]]: def _findings_for_evidence(run_id: str, evidence: list[dict[str, Any]]) -> list[dict[str, Any]]:

View File

@@ -0,0 +1,224 @@
"""Normalizer plug-in bridge for extension-provided runner output."""
from __future__ import annotations
import importlib.util
from pathlib import Path
from types import ModuleType
from typing import Any
from guide_board.errors import ValidationError
from guide_board.io import load_json
def normalize_step_result(
root: Path,
run_dir: Path,
run_id: str,
plan: dict[str, Any],
step: dict[str, Any],
runner_result: dict[str, Any],
) -> dict[str, Any]:
"""Apply matching extension normalizers to a runner result."""
extension = _extension_snapshot(plan, step["extension_id"])
extension_path = _snapshot_path(root, extension)
manifest = load_json(extension_path / "extension.json")
result = _coerce_result(runner_result)
applied: list[str] = []
for normalizer in _matching_normalizers(manifest, step):
normalized = _run_normalizer(
root,
run_dir,
run_id,
plan,
step,
extension_path,
normalizer,
result,
)
if _is_normalizer_error(normalized):
return normalized
result = _merge_result(result, normalized)
applied.append(normalizer["id"])
if applied:
facts = dict(result.get("facts", {}))
facts["normalizer_refs"] = applied
result["facts"] = facts
return result
def _matching_normalizers(
manifest: dict[str, Any],
step: dict[str, Any],
) -> list[dict[str, Any]]:
matching = []
runner_ref = step.get("runner_ref")
for normalizer in manifest.get("normalizers", []):
if not isinstance(normalizer, dict):
continue
normalizer_runner_ref = normalizer.get("runner_ref")
if normalizer_runner_ref and normalizer_runner_ref != runner_ref:
continue
matching.append(normalizer)
return matching
def _run_normalizer(
root: Path,
run_dir: Path,
run_id: str,
plan: dict[str, Any],
step: dict[str, Any],
extension_path: Path,
normalizer: dict[str, Any],
runner_result: dict[str, Any],
) -> dict[str, Any]:
if normalizer["kind"] != "python_module":
raise ValidationError(
f"{normalizer['id']}: unsupported normalizer kind {normalizer['kind']!r}"
)
module_path = normalizer.get("module_path")
callable_name = normalizer.get("callable")
if not module_path or not callable_name:
raise ValidationError(
f"{normalizer['id']}: python_module normalizers need module_path and callable"
)
module_file = (extension_path / module_path).resolve()
try:
module_file.relative_to(extension_path.resolve())
except ValueError as exc:
raise ValidationError(
f"{normalizer['id']}: module_path must stay inside the extension directory"
) from exc
module = _load_module(module_file, normalizer["id"])
normalizer_callable = getattr(module, callable_name, None)
if not callable(normalizer_callable):
raise ValidationError(f"{normalizer['id']}: callable {callable_name!r} was not found")
context = {
"root": str(root),
"run_dir": str(run_dir),
"run_id": run_id,
"plan": plan,
"step": step,
"target_profile": plan["target_profile_snapshot"],
"assessment_profile": plan["assessment_profile_snapshot"],
"extension_path": str(extension_path),
"normalizer": normalizer,
"runner_result": runner_result,
}
try:
result = normalizer_callable(context)
except Exception as exc: # noqa: BLE001 - extension failures become evidence.
return {
"result": "infrastructure_error",
"observations": [
f"Normalizer {normalizer['id']!r} failed before producing evidence: {exc}"
],
"facts": {
"normalizer_ref": normalizer["id"],
"normalizer_kind": normalizer["kind"],
"error_type": type(exc).__name__,
},
"artifact_refs": runner_result.get("artifact_refs", []),
"requirement_refs": runner_result.get("requirement_refs", []),
}
if not isinstance(result, dict):
raise ValidationError(f"{normalizer['id']}: normalizer must return an object")
return result
def _merge_result(
base: dict[str, Any],
update: dict[str, Any],
) -> dict[str, Any]:
merged = dict(base)
if "result" in update:
merged["result"] = update["result"]
if "observations" in update:
merged["observations"] = _string_list(base.get("observations", []))
merged["observations"].extend(_string_list(update.get("observations", [])))
if "facts" in update:
facts = dict(base.get("facts", {}))
update_facts = update.get("facts", {})
if isinstance(update_facts, dict):
facts.update(update_facts)
merged["facts"] = facts
if "artifact_refs" in update:
merged["artifact_refs"] = _dedupe(
_string_list(base.get("artifact_refs", []))
+ _string_list(update.get("artifact_refs", []))
)
if "requirement_refs" in update:
merged["requirement_refs"] = _dedupe(
_string_list(base.get("requirement_refs", []))
+ _string_list(update.get("requirement_refs", []))
)
return _coerce_result(merged)
def _coerce_result(value: dict[str, Any]) -> dict[str, Any]:
facts = value.get("facts", {})
if not isinstance(facts, dict):
facts = {}
return {
"result": value.get("result", "unknown"),
"observations": _string_list(value.get("observations", [])),
"facts": facts,
"artifact_refs": _string_list(value.get("artifact_refs", [])),
"requirement_refs": _string_list(value.get("requirement_refs", [])),
}
def _is_normalizer_error(result: dict[str, Any]) -> bool:
return (
result.get("result") == "infrastructure_error"
and "normalizer_ref" in result.get("facts", {})
)
def _string_list(value: Any) -> list[str]:
if not isinstance(value, list):
return []
return [item for item in value if isinstance(item, str)]
def _dedupe(values: list[str]) -> list[str]:
seen = set()
deduped = []
for value in values:
if value in seen:
continue
seen.add(value)
deduped.append(value)
return deduped
def _load_module(path: Path, normalizer_id: str) -> ModuleType:
if not path.exists():
raise ValidationError(f"{normalizer_id}: module not found: {path}")
module_name = f"_guide_board_normalizer_{normalizer_id.replace('-', '_')}"
spec = importlib.util.spec_from_file_location(module_name, path)
if spec is None or spec.loader is None:
raise ValidationError(f"{normalizer_id}: unable to load module from {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _extension_snapshot(plan: dict[str, Any], extension_id: str) -> dict[str, Any]:
for extension in plan["extension_snapshots"]:
if extension["id"] == extension_id:
return extension
raise ValidationError(f"step references unknown extension {extension_id!r}")
def _snapshot_path(root: Path, extension: dict[str, Any]) -> Path:
path = Path(extension["path"])
return path if path.is_absolute() else root / path

View File

@@ -2,6 +2,7 @@ from __future__ import annotations
import http.client import http.client
import json import json
import shutil
import time import time
import unittest import unittest
from tempfile import TemporaryDirectory from tempfile import TemporaryDirectory
@@ -193,6 +194,52 @@ class CoreArchitectureTests(unittest.TestCase):
): ):
validate_target_profile(target_path, extensions) validate_target_profile(target_path, extensions)
def test_runs_sdk_fixture_from_external_extension_repo(self) -> None:
with TemporaryDirectory() as temporary_directory:
temp_root = Path(temporary_directory)
extension_dir = temp_root / "sdk-fixture"
shutil.copytree(ROOT / "extensions" / "sdk-fixture", extension_dir)
result = run_assessment(
temp_root,
extension_dir / "profiles" / "targets" / "sdk-fixture-target.json",
extension_dir / "profiles" / "assessments" / "sdk-fixture-assessment.json",
temp_root / "runs" / "sdk-fixture",
[extension_dir],
)
run_dir = Path(result["run_dir"])
plan = json.loads((run_dir / "plan.json").read_text(encoding="utf-8"))
evidence = json.loads(
(run_dir / "normalized" / "evidence.json").read_text(encoding="utf-8")
)["evidence"]
mappings = json.loads(
(run_dir / "normalized" / "mappings.json").read_text(encoding="utf-8")
)["mappings"]
assessment_package = json.loads(
(run_dir / "reports" / "assessment-package.json").read_text(encoding="utf-8")
)
self.assertEqual(result["status"], "completed")
self.assertEqual(plan["extension_snapshots"][0]["source"], "external")
self.assertEqual(plan["target_profile_snapshot"]["subject_type"], "sdk-fixture-target")
self.assertEqual([item["result"] for item in evidence], ["skipped", "pass"])
check_evidence = evidence[1]
self.assertEqual(
check_evidence["facts"]["normalizer_refs"],
["native-probe-normalizer"],
)
self.assertEqual(check_evidence["facts"]["native_score"], 98)
self.assertEqual(
check_evidence["requirement_refs"],
["guide-board.sdk-fixture.v1.native-output"],
)
self.assertEqual(
check_evidence["artifact_refs"],
["artifacts/sdk-fixture/native-result.json"],
)
self.assertEqual(mappings[0]["target_id"], "normalizer-plugin")
self.assertEqual(assessment_package["summary"], {"pass": 1, "skipped": 1})
def test_runs_sample_noop_assessment(self) -> None: def test_runs_sample_noop_assessment(self) -> None:
with TemporaryDirectory() as temporary_directory: with TemporaryDirectory() as temporary_directory:
result = run_assessment( result = run_assessment(

View File

@@ -4,7 +4,7 @@ type: workplan
title: "Extension SDK Maturity" title: "Extension SDK Maturity"
repo: guide-board repo: guide-board
domain: markitect domain: markitect
status: active status: completed
owner: codex owner: codex
planning_priority: high planning_priority: high
planning_order: 3 planning_order: 3
@@ -69,7 +69,7 @@ Progress:
```task ```task
id: GUIDE-BOARD-WP-0003-T002 id: GUIDE-BOARD-WP-0003-T002
status: todo status: done
priority: high priority: high
state_hub_task_id: "b87e68c1-6eca-4274-8e3f-6e2854c5a1e1" state_hub_task_id: "b87e68c1-6eca-4274-8e3f-6e2854c5a1e1"
``` ```
@@ -81,11 +81,22 @@ Acceptance:
normalize native result artifacts explicitly. normalize native result artifacts explicitly.
- Add tests that prove a normalizer can map native output into evidence. - Add tests that prove a normalizer can map native output into evidence.
Progress:
- Added `guide_board.normalizers.normalize_step_result`.
- Extended `normalizers` manifest entries to support Python module descriptor
objects while preserving the existing string shorthand.
- Invoked matching normalizers after runner execution and before evidence
writing.
- Merged normalizer result fields over runner results and recorded
`normalizer_refs` in evidence facts.
- Added test coverage through the SDK fixture run.
## D3.3 - SDK Fixture Extension And Acceptance Tests ## D3.3 - SDK Fixture Extension And Acceptance Tests
```task ```task
id: GUIDE-BOARD-WP-0003-T003 id: GUIDE-BOARD-WP-0003-T003
status: todo status: done
priority: medium priority: medium
state_hub_task_id: "f3738751-5a0d-4eaf-85b1-75e599a78060" state_hub_task_id: "f3738751-5a0d-4eaf-85b1-75e599a78060"
``` ```
@@ -97,11 +108,19 @@ Acceptance:
- Cover external repo discovery, schema validation, normalizer invocation, plan - Cover external repo discovery, schema validation, normalizer invocation, plan
generation, and result package shape. generation, and result package shape.
Progress:
- Added `extensions/sdk-fixture`.
- Included extension-owned target and assessment schemas, fixture profiles, a
native-output runner, a normalizer, and a mapping set.
- Added a unit test that copies the fixture as an external extension repository
and verifies plan, evidence, mapping, and assessment package output.
## D3.4 - Extension Authoring Documentation Refresh ## D3.4 - Extension Authoring Documentation Refresh
```task ```task
id: GUIDE-BOARD-WP-0003-T004 id: GUIDE-BOARD-WP-0003-T004
status: todo status: done
priority: medium priority: medium
state_hub_task_id: "3d390bd4-755b-462a-9e16-9c859990d99e" state_hub_task_id: "3d390bd4-755b-462a-9e16-9c859990d99e"
``` ```
@@ -114,6 +133,14 @@ Acceptance:
- Link the SDK maturity guidance from the assessment operations and external - Link the SDK maturity guidance from the assessment operations and external
extension acceptance docs where useful. extension acceptance docs where useful.
Progress:
- Refreshed `docs/EXTENSION-SDK.md` with profile schema descriptors,
normalizer descriptors, context fields, merge semantics, and fixture guidance.
- Linked SDK authoring contracts from `docs/ASSESSMENT-OPERATIONS.md`.
- Linked `extensions/sdk-fixture` from `docs/EXTERNAL-EXTENSION-ACCEPTANCE.md`.
- Added README references for the SDK fixture and WP3.
## Definition Of Done ## Definition Of Done
- External extension repositories can declare and test domain-specific profile - External extension repositories can declare and test domain-specific profile