first extension execution path

This commit is contained in:
2026-05-07 12:24:38 +02:00
parent 3707f01f39
commit 228193723a
13 changed files with 697 additions and 27 deletions

View File

@@ -32,6 +32,7 @@ See:
- [INTENT.md](INTENT.md)
- [docs/ARCHITECTURE-BLUEPRINT.md](docs/ARCHITECTURE-BLUEPRINT.md)
- [docs/EXTENSION-SDK.md](docs/EXTENSION-SDK.md)
- [extensions/CANDIDATES.md](extensions/CANDIDATES.md)
- [extensions/open-cmis-tck/INTENT.md](extensions/open-cmis-tck/INTENT.md)
- [workplans/GUIDE-BOARD-WP-0001-bootstrapping.md](workplans/GUIDE-BOARD-WP-0001-bootstrapping.md)

View File

@@ -771,3 +771,5 @@ Each run should lock:
7. Add optional service API around the CLI job model.
8. Add OSCAL export and procedural evidence-pack support after the internal
evidence model proves itself with executable extensions.
The first extension SDK contract is documented in `docs/EXTENSION-SDK.md`.

145
docs/EXTENSION-SDK.md Normal file
View File

@@ -0,0 +1,145 @@
# Guide Board Extension SDK
Status: draft
Created: 2026-05-07
## Purpose
This document defines the first extension integration contract for `guide-board`.
It is intentionally small: extensions declare metadata in `extension.json`, the
core discovers them, and runners can produce normalized evidence through a stable
dictionary contract.
## Extension Layout
Incubating extensions live under:
```text
extensions/<extension-id>/
INTENT.md
extension.json
src/
docs/
schemas/
checks/
mappings/
profiles/
runners/
normalizers/
reports/
workplans/
```
Only `INTENT.md` and `extension.json` are required for discovery. Additional
folders appear as the extension grows.
## Manifest Contract
`extension.json` must validate against:
```text
docs/schemas/extension-manifest.schema.json
```
The key runtime fields are:
- `id`: must match the extension directory name.
- `extension_type`: one of the supported archetypes from the architecture
blueprint.
- `supported_frameworks`: framework IDs this extension can contribute evidence
for.
- `check_groups`: named groups that assessment profiles can select.
- `preflight_runner`: optional runner ID used before selected check groups.
- `runner_entrypoints`: concrete runner declarations.
- `certification_boundary`: explicit statement of what the extension does not
certify.
## Runner Entry Points
Runner entry points currently support these kinds:
- `python_module`: load a Python file from the extension directory and call a
function.
- `external`: declare an external harness that the baseline core cannot execute
yet.
- `command`: reserved for future command execution.
Example:
```json
{
"id": "cmis-browser-preflight",
"kind": "python_module",
"module_path": "src/open_cmis_tck/preflight.py",
"callable": "run",
"command": null,
"description": "Checks whether the CMIS Browser Binding endpoint is reachable."
}
```
## Python Runner Contract
A Python runner receives one context object and returns one result object.
```python
def run(context: dict) -> dict:
return {
"result": "pass",
"observations": ["Observed the expected condition."],
"facts": {"key": "value"},
"artifact_refs": [],
}
```
Context fields:
- `root`: repository root path as a string.
- `run_dir`: output run directory path as a string.
- `run_id`: current run ID.
- `plan`: full run plan snapshot.
- `step`: the step being executed.
- `target_profile`: target profile snapshot.
- `assessment_profile`: assessment profile snapshot.
- `extension_path`: extension directory path as a string.
- `runner`: manifest runner declaration.
Result fields:
- `result`: one of the guide-board evidence result statuses.
- `observations`: human-readable observations.
- `facts`: structured facts extracted by the runner.
- `artifact_refs`: references to raw artifacts written by the runner.
If a Python runner raises an exception, the core converts that failure into
`infrastructure_error` evidence so the assessment package remains complete.
## Result Statuses
Initial statuses:
- `pass`
- `fail`
- `warning`
- `manual`
- `not_applicable`
- `skipped`
- `expected_gap`
- `waiver_applied`
- `unsupported_by_design`
- `infrastructure_error`
- `blocked`
- `unknown`
## Current Extension Examples
- `sample-noop`: no runner, used to validate the core contracts.
- `open-cmis-tck`: provides a Python CMIS Browser Binding preflight runner and
declares the future external OpenCMIS TCK runner.
## Next SDK Steps
- Add command runner support with explicit allow/deny controls.
- Add artifact helper APIs for extension-generated raw files.
- Add normalizer and mapping plug-in contracts.
- Add extension-owned schema validation for domain-specific target profile
fields.

View File

@@ -60,7 +60,25 @@
}
},
"preflight_runner": { "type": ["string", "null"] },
"runner_entrypoints": { "type": "array", "items": { "type": "string" } },
"runner_entrypoints": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["id", "kind"],
"properties": {
"id": { "type": "string" },
"kind": {
"type": "string",
"enum": ["python_module", "command", "external"]
},
"module_path": { "type": ["string", "null"] },
"callable": { "type": ["string", "null"] },
"command": { "type": ["array", "null"], "items": { "type": "string" } },
"description": { "type": ["string", "null"] }
}
}
},
"normalizers": { "type": "array", "items": { "type": "string" } },
"mappings": { "type": "array", "items": { "type": "string" } },
"report_fragments": { "type": "array", "items": { "type": "string" } },

View File

@@ -12,7 +12,16 @@
],
"check_groups": [],
"preflight_runner": null,
"runner_entrypoints": [],
"runner_entrypoints": [
{
"id": "replace-with-runner-id",
"kind": "external",
"module_path": null,
"callable": null,
"command": null,
"description": "Describe how this runner is provided."
}
],
"normalizers": [],
"mappings": [],
"report_fragments": [],

View File

@@ -59,7 +59,22 @@
],
"preflight_runner": "cmis-browser-preflight",
"runner_entrypoints": [
"opencmis-tck"
{
"id": "cmis-browser-preflight",
"kind": "python_module",
"module_path": "src/open_cmis_tck/preflight.py",
"callable": "run",
"command": null,
"description": "Checks whether the configured CMIS Browser Binding endpoint is reachable and returns parseable repository metadata."
},
{
"id": "opencmis-tck",
"kind": "external",
"module_path": null,
"callable": null,
"command": null,
"description": "Placeholder for the Java/Maven Apache Chemistry OpenCMIS TCK runner."
}
],
"normalizers": [
"opencmis-result-normalizer"

View File

@@ -0,0 +1,161 @@
"""CMIS Browser Binding preflight runner."""
from __future__ import annotations
import json
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
def run(context: dict[str, Any]) -> dict[str, Any]:
target = context["target_profile"]
endpoint = _browser_endpoint(target)
if endpoint is None:
return {
"result": "fail",
"observations": [
"Target profile does not declare a CMIS Browser Binding endpoint."
],
"facts": {
"endpoint_found": False,
},
"artifact_refs": [],
}
timeout = _timeout_seconds(context)
request = Request(
endpoint["url"],
headers={
"Accept": "application/json, */*;q=0.1",
"User-Agent": "guide-board-open-cmis-tck-preflight/0.1.0",
},
)
try:
with urlopen(request, timeout=timeout) as response:
status_code = response.status
content_type = response.headers.get("Content-Type", "")
body = response.read(1024 * 1024)
except HTTPError as exc:
return {
"result": "infrastructure_error",
"observations": [
f"CMIS Browser Binding endpoint returned HTTP {exc.code}."
],
"facts": {
"endpoint_found": True,
"url": endpoint["url"],
"http_status": exc.code,
},
"artifact_refs": [],
}
except URLError as exc:
return {
"result": "infrastructure_error",
"observations": [
f"CMIS Browser Binding endpoint is not reachable: {exc.reason}."
],
"facts": {
"endpoint_found": True,
"url": endpoint["url"],
"error": str(exc.reason),
},
"artifact_refs": [],
}
except TimeoutError:
return {
"result": "infrastructure_error",
"observations": [
f"CMIS Browser Binding endpoint did not respond within {timeout} seconds."
],
"facts": {
"endpoint_found": True,
"url": endpoint["url"],
"timeout_seconds": timeout,
},
"artifact_refs": [],
}
facts: dict[str, Any] = {
"endpoint_found": True,
"url": endpoint["url"],
"binding": endpoint["binding"],
"http_status": status_code,
"content_type": content_type,
}
parsed = _parse_json(body)
if parsed is None:
facts["json_detected"] = False
return {
"result": "warning",
"observations": [
"CMIS Browser Binding endpoint is reachable but did not return parseable JSON."
],
"facts": facts,
"artifact_refs": [],
}
facts["json_detected"] = True
facts.update(_repository_facts(parsed))
return {
"result": "pass",
"observations": [
"CMIS Browser Binding endpoint is reachable and returned parseable JSON."
],
"facts": facts,
"artifact_refs": [],
}
def _browser_endpoint(target: dict[str, Any]) -> dict[str, Any] | None:
for endpoint in target.get("endpoints", []):
if endpoint.get("binding") == "cmis-browser":
return endpoint
return None
def _timeout_seconds(context: dict[str, Any]) -> float:
runtime_policy = context["assessment_profile"].get("runtime_policy", {})
configured = runtime_policy.get("timeout_seconds", 5)
if not isinstance(configured, (int, float)):
return 5.0
return max(1.0, min(float(configured), 10.0))
def _parse_json(body: bytes) -> Any:
try:
return json.loads(body.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError):
return None
def _repository_facts(value: Any) -> dict[str, Any]:
if not isinstance(value, dict):
return {"repository_shape": "unknown"}
if "repositoryId" in value:
return {
"repository_shape": "single-repository-info",
"repository_ids": [value["repositoryId"]],
"cmis_version_supported": value.get("cmisVersionSupported"),
"capabilities_present": isinstance(value.get("capabilities"), dict),
}
repository_ids = []
for key, child in value.items():
if isinstance(child, dict) and (
"repositoryId" in child or "repositoryName" in child
):
repository_ids.append(str(child.get("repositoryId", key)))
if repository_ids:
return {
"repository_shape": "repository-map",
"repository_ids": repository_ids,
}
return {
"repository_shape": "object",
"top_level_keys": sorted(str(key) for key in value.keys())[:20],
}

View File

@@ -91,7 +91,7 @@ Acceptance:
```task
id: OPEN-CMIS-TCK-WP-0001-T003
status: todo
status: in_progress
priority: high
state_hub_task_id: "6d45885b-78a4-4e8b-8fcc-b8d6488e703b"
```
@@ -103,6 +103,13 @@ Acceptance:
- Unsupported optional capabilities can be accepted as expected gaps.
- Preflight output is captured as structured JSON.
Progress:
- The first CMIS Browser Binding preflight runner checks endpoint reachability
and parseable JSON repository metadata through the guide-board runner bridge.
- Capability flag normalization remains to be expanded after a live target sample
is captured.
## D1.4 - OpenCMIS TCK Runner Wrapper
```task

View File

@@ -9,6 +9,7 @@ from typing import Any
from guide_board.io import write_json
from guide_board.planning import build_run_plan
from guide_board.runners import run_step
from guide_board.schema import assert_valid
@@ -23,7 +24,10 @@ def run_assessment(
run_dir = output_dir or root / "runs" / run_id
created_at = _now()
evidence = [_evidence_for_step(run_id, plan, step) for step in plan["ordered_steps"]]
evidence = [
_evidence_for_step(root, run_dir, run_id, plan, step)
for step in plan["ordered_steps"]
]
for item in evidence:
assert_valid(item, "evidence-item")
@@ -53,19 +57,16 @@ def run_assessment(
}
def _evidence_for_step(run_id: str, plan: dict[str, Any], step: dict[str, Any]) -> dict[str, Any]:
def _evidence_for_step(
root: Path,
run_dir: Path,
run_id: str,
plan: dict[str, Any],
step: dict[str, Any],
) -> dict[str, Any]:
now = _now()
runner_ref = step.get("runner_ref")
if runner_ref is None:
result = "manual" if step["kind"] == "check_group" else "skipped"
observations = [
"No runner is configured for this step in the baseline core."
]
else:
result = "blocked"
observations = [
f"Runner {runner_ref!r} is declared but not implemented by the baseline core."
]
runner_result = run_step(root, run_dir, run_id, plan, step)
return {
"id": f"evidence:{step['id']}",
@@ -73,14 +74,15 @@ def _evidence_for_step(run_id: str, plan: dict[str, Any], step: dict[str, Any])
"extension_id": step["extension_id"],
"check_id": step["id"],
"subject_ref": plan["target_profile_snapshot"]["id"],
"result": result,
"observations": observations,
"result": runner_result["result"],
"observations": runner_result["observations"],
"facts": {
"step_kind": step["kind"],
"runner_ref": runner_ref,
**runner_result["facts"],
},
"requirement_refs": _requirement_refs(plan, step),
"artifact_refs": [],
"artifact_refs": runner_result["artifact_refs"],
"started_at": now,
"completed_at": now,
}
@@ -95,25 +97,38 @@ def _requirement_refs(plan: dict[str, Any], step: dict[str, Any]) -> list[str]:
def _findings_for_evidence(run_id: str, evidence: list[dict[str, Any]]) -> list[dict[str, Any]]:
findings: list[dict[str, Any]] = []
for item in evidence:
if item["result"] != "blocked":
if item["result"] not in {"blocked", "fail", "infrastructure_error"}:
continue
classification = {
"blocked": "runner_not_implemented",
"fail": "check_failed",
"infrastructure_error": "infrastructure_error",
}[item["result"]]
findings.append(
{
"id": f"finding:{item['check_id']}",
"run_id": run_id,
"status": "blocked",
"severity": "info",
"classification": "runner_not_implemented",
"status": item["result"],
"severity": "info" if item["result"] == "blocked" else "medium",
"classification": classification,
"requirement_refs": item["requirement_refs"],
"evidence_refs": [item["id"]],
"expected": True,
"expected": item["result"] == "blocked",
"waiver_ref": None,
"remediation": "Implement or configure the declared extension runner.",
"remediation": _remediation_for_result(item["result"]),
}
)
return findings
def _remediation_for_result(result: str) -> str:
if result == "blocked":
return "Implement or configure the declared extension runner."
if result == "infrastructure_error":
return "Fix the target, network, credentials, or harness runtime and rerun the assessment."
return "Review the failed check and target implementation."
def _assessment_package(
run_id: str,
plan: dict[str, Any],
@@ -198,6 +213,8 @@ def _markdown_report(run_metadata: dict[str, Any], package: dict[str, Any]) -> s
def _run_status(evidence: list[dict[str, Any]]) -> str:
if any(item["result"] == "fail" for item in evidence):
return "failed"
if any(item["result"] == "infrastructure_error" for item in evidence):
return "infrastructure_error"
if any(item["result"] == "blocked" for item in evidence):
return "blocked"
return "completed"

162
src/guide_board/runners.py Normal file
View File

@@ -0,0 +1,162 @@
"""Runner bridge for extension-provided checks."""
from __future__ import annotations
import importlib.util
from pathlib import Path
from types import ModuleType
from typing import Any, Callable
from guide_board.errors import ValidationError
from guide_board.io import load_json
RunnerCallable = Callable[[dict[str, Any]], dict[str, Any]]
def run_step(
root: Path,
run_dir: Path,
run_id: str,
plan: dict[str, Any],
step: dict[str, Any],
) -> dict[str, Any]:
runner_ref = step.get("runner_ref")
if runner_ref is None:
return _no_runner_result(step)
extension = _extension_snapshot(plan, step["extension_id"])
extension_path = root / extension["path"]
manifest = load_json(extension_path / "extension.json")
entrypoint = _runner_entrypoint(manifest, runner_ref)
if entrypoint["kind"] == "python_module":
return _run_python_module(root, run_dir, run_id, plan, step, extension_path, entrypoint)
if entrypoint["kind"] == "external":
return {
"result": "blocked",
"observations": [
f"Runner {runner_ref!r} is declared as an external runner and is not implemented by the core."
],
"facts": {
"runner_ref": runner_ref,
"runner_kind": "external",
},
"artifact_refs": [],
}
if entrypoint["kind"] == "command":
return {
"result": "blocked",
"observations": [
f"Runner {runner_ref!r} is declared as a command runner; command execution is not enabled yet."
],
"facts": {
"runner_ref": runner_ref,
"runner_kind": "command",
},
"artifact_refs": [],
}
raise ValidationError(f"{runner_ref}: unsupported runner kind {entrypoint['kind']!r}")
def _no_runner_result(step: dict[str, Any]) -> dict[str, Any]:
result = "manual" if step["kind"] == "check_group" else "skipped"
return {
"result": result,
"observations": [
"No runner is configured for this step in the baseline core."
],
"facts": {
"runner_ref": None,
"runner_kind": None,
},
"artifact_refs": [],
}
def _run_python_module(
root: Path,
run_dir: Path,
run_id: str,
plan: dict[str, Any],
step: dict[str, Any],
extension_path: Path,
entrypoint: dict[str, Any],
) -> dict[str, Any]:
module_path = entrypoint.get("module_path")
callable_name = entrypoint.get("callable")
if not module_path or not callable_name:
raise ValidationError(f"{entrypoint['id']}: python_module runners need module_path and callable")
module_file = (extension_path / module_path).resolve()
try:
module_file.relative_to(extension_path.resolve())
except ValueError as exc:
raise ValidationError(
f"{entrypoint['id']}: module_path must stay inside the extension directory"
) from exc
module = _load_module(module_file, entrypoint["id"])
runner = getattr(module, callable_name, None)
if not callable(runner):
raise ValidationError(f"{entrypoint['id']}: callable {callable_name!r} was not found")
context = {
"root": str(root),
"run_dir": str(run_dir),
"run_id": run_id,
"plan": plan,
"step": step,
"target_profile": plan["target_profile_snapshot"],
"assessment_profile": plan["assessment_profile_snapshot"],
"extension_path": str(extension_path),
"runner": entrypoint,
}
try:
result = runner(context)
except Exception as exc: # noqa: BLE001 - extension failures become evidence.
return {
"result": "infrastructure_error",
"observations": [
f"Runner {entrypoint['id']!r} failed before producing evidence: {exc}"
],
"facts": {
"runner_ref": entrypoint["id"],
"runner_kind": "python_module",
"error_type": type(exc).__name__,
},
"artifact_refs": [],
}
if not isinstance(result, dict):
raise ValidationError(f"{entrypoint['id']}: runner must return an object")
return {
"result": result.get("result", "unknown"),
"observations": result.get("observations", []),
"facts": result.get("facts", {}),
"artifact_refs": result.get("artifact_refs", []),
}
def _load_module(path: Path, runner_id: str) -> ModuleType:
if not path.exists():
raise ValidationError(f"{runner_id}: module not found: {path}")
module_name = f"_guide_board_runner_{runner_id.replace('-', '_')}"
spec = importlib.util.spec_from_file_location(module_name, path)
if spec is None or spec.loader is None:
raise ValidationError(f"{runner_id}: unable to load module from {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _extension_snapshot(plan: dict[str, Any], extension_id: str) -> dict[str, Any]:
for extension in plan["extension_snapshots"]:
if extension["id"] == extension_id:
return extension
raise ValidationError(f"step references unknown extension {extension_id!r}")
def _runner_entrypoint(manifest: dict[str, Any], runner_ref: str) -> dict[str, Any]:
for entrypoint in manifest.get("runner_entrypoints", []):
if entrypoint["id"] == runner_ref:
return entrypoint
raise ValidationError(f"{manifest['id']}: runner {runner_ref!r} is not declared")

16
src/guide_board/sdk.py Normal file
View File

@@ -0,0 +1,16 @@
"""Public helper types for extension runners.
Extension Python runners are called with one dictionary context and should return
one dictionary shaped like `RunnerResult`.
"""
from __future__ import annotations
from typing import Any, TypedDict
class RunnerResult(TypedDict, total=False):
result: str
observations: list[str]
facts: dict[str, Any]
artifact_refs: list[str]

View File

@@ -1,6 +1,9 @@
from __future__ import annotations
import unittest
import json
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer
from tempfile import TemporaryDirectory
from pathlib import Path
@@ -79,6 +82,111 @@ class CoreArchitectureTests(unittest.TestCase):
self.assertTrue((run_dir / "reports" / "assessment-package.json").exists())
self.assertTrue((run_dir / "reports" / "report.md").exists())
def test_runs_cmis_preflight_against_local_endpoint(self) -> None:
server = HTTPServer(("127.0.0.1", 0), _CmisHandler)
thread = threading.Thread(target=server.serve_forever)
thread.daemon = True
thread.start()
try:
with TemporaryDirectory() as temporary_directory:
temp_root = Path(temporary_directory)
target_path = temp_root / "target.json"
assessment_path = temp_root / "assessment.json"
target_path.write_text(
json.dumps(
{
"id": "local-cmis-test",
"subject_type": "cmis-browser-binding-endpoint",
"subject_name": "Local CMIS Test",
"environment": "test",
"scope": ["preflight"],
"endpoints": [
{
"id": "browser-binding",
"url": f"http://127.0.0.1:{server.server_port}/cmis/browser",
"binding": "cmis-browser",
}
],
"artifacts": [],
"credentials_ref": None,
"declared_capabilities": ["cmis.repository-info"],
"known_gaps": [],
}
),
encoding="utf-8",
)
assessment_path.write_text(
json.dumps(
{
"id": "local-cmis-preflight",
"framework_refs": ["cmis.browser-binding.compatibility.v1"],
"extension_refs": ["open-cmis-tck"],
"target_profile_ref": "local-cmis-test",
"selected_check_groups": {"open-cmis-tck": []},
"expectations_ref": None,
"waivers_ref": None,
"output_policy": {
"report_formats": ["json", "markdown"],
"artifact_retention": "summary-only",
},
"retention_policy": {
"summary_days": 365,
"raw_artifact_days": 0,
},
"runtime_policy": {
"offline": False,
"timeout_seconds": 2,
},
}
),
encoding="utf-8",
)
result = run_assessment(
ROOT,
target_path,
assessment_path,
temp_root / "run",
)
evidence = json.loads(
(Path(result["run_dir"]) / "normalized" / "evidence.json").read_text(
encoding="utf-8"
)
)
self.assertEqual(result["status"], "completed")
self.assertEqual(evidence["evidence"][0]["result"], "pass")
self.assertEqual(
evidence["evidence"][0]["facts"]["repository_ids"],
["local-test-repository"],
)
finally:
server.shutdown()
thread.join(timeout=5)
server.server_close()
class _CmisHandler(BaseHTTPRequestHandler):
def do_GET(self) -> None:
body = json.dumps(
{
"local-test-repository": {
"repositoryId": "local-test-repository",
"repositoryName": "Local Test Repository",
"cmisVersionSupported": "1.1",
"capabilities": {},
}
}
).encode("utf-8")
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, format: str, *args: object) -> None:
return
if __name__ == "__main__":
unittest.main()

View File

@@ -189,7 +189,7 @@ Acceptance:
```task
id: GUIDE-BOARD-WP-0001-T006
status: todo
status: done
priority: high
state_hub_task_id: "3c757929-a5e4-4c11-bbf1-6d7f26def93e"
```
@@ -201,12 +201,13 @@ Acceptance:
- Provide a minimal extension template.
- Extension ownership boundaries make later extraction to a separate repository
straightforward.
- Python module runner contracts are documented in `docs/EXTENSION-SDK.md`.
## D1.8 - CMIS Seed Extension Integration
```task
id: GUIDE-BOARD-WP-0001-T007
status: todo
status: in_progress
priority: high
state_hub_task_id: "455f92b0-1d2b-43d0-aa61-464d9dc83a62"
```
@@ -218,6 +219,14 @@ Acceptance:
- CMIS output normalizes into the same evidence model used by other extensions.
- CMIS capability mappings are extension-owned.
Progress:
- `open-cmis-tck` declares runner entry points through `extension.json`.
- The CMIS Browser Binding preflight runner executes through the generic runner
bridge and produces normalized evidence.
- The OpenCMIS Java/Maven TCK runner remains external and blocked until the
extension harness wrapper is implemented.
## D1.9 - Containerized Execution Design
```task