generated from coulomb/repo-seed
first extension execution path
This commit is contained in:
@@ -32,6 +32,7 @@ See:
|
|||||||
|
|
||||||
- [INTENT.md](INTENT.md)
|
- [INTENT.md](INTENT.md)
|
||||||
- [docs/ARCHITECTURE-BLUEPRINT.md](docs/ARCHITECTURE-BLUEPRINT.md)
|
- [docs/ARCHITECTURE-BLUEPRINT.md](docs/ARCHITECTURE-BLUEPRINT.md)
|
||||||
|
- [docs/EXTENSION-SDK.md](docs/EXTENSION-SDK.md)
|
||||||
- [extensions/CANDIDATES.md](extensions/CANDIDATES.md)
|
- [extensions/CANDIDATES.md](extensions/CANDIDATES.md)
|
||||||
- [extensions/open-cmis-tck/INTENT.md](extensions/open-cmis-tck/INTENT.md)
|
- [extensions/open-cmis-tck/INTENT.md](extensions/open-cmis-tck/INTENT.md)
|
||||||
- [workplans/GUIDE-BOARD-WP-0001-bootstrapping.md](workplans/GUIDE-BOARD-WP-0001-bootstrapping.md)
|
- [workplans/GUIDE-BOARD-WP-0001-bootstrapping.md](workplans/GUIDE-BOARD-WP-0001-bootstrapping.md)
|
||||||
|
|||||||
@@ -771,3 +771,5 @@ Each run should lock:
|
|||||||
7. Add optional service API around the CLI job model.
|
7. Add optional service API around the CLI job model.
|
||||||
8. Add OSCAL export and procedural evidence-pack support after the internal
|
8. Add OSCAL export and procedural evidence-pack support after the internal
|
||||||
evidence model proves itself with executable extensions.
|
evidence model proves itself with executable extensions.
|
||||||
|
|
||||||
|
The first extension SDK contract is documented in `docs/EXTENSION-SDK.md`.
|
||||||
|
|||||||
145
docs/EXTENSION-SDK.md
Normal file
145
docs/EXTENSION-SDK.md
Normal file
@@ -0,0 +1,145 @@
|
|||||||
|
# Guide Board Extension SDK
|
||||||
|
|
||||||
|
Status: draft
|
||||||
|
Created: 2026-05-07
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This document defines the first extension integration contract for `guide-board`.
|
||||||
|
It is intentionally small: extensions declare metadata in `extension.json`, the
|
||||||
|
core discovers them, and runners can produce normalized evidence through a stable
|
||||||
|
dictionary contract.
|
||||||
|
|
||||||
|
## Extension Layout
|
||||||
|
|
||||||
|
Incubating extensions live under:
|
||||||
|
|
||||||
|
```text
|
||||||
|
extensions/<extension-id>/
|
||||||
|
INTENT.md
|
||||||
|
extension.json
|
||||||
|
src/
|
||||||
|
docs/
|
||||||
|
schemas/
|
||||||
|
checks/
|
||||||
|
mappings/
|
||||||
|
profiles/
|
||||||
|
runners/
|
||||||
|
normalizers/
|
||||||
|
reports/
|
||||||
|
workplans/
|
||||||
|
```
|
||||||
|
|
||||||
|
Only `INTENT.md` and `extension.json` are required for discovery. Additional
|
||||||
|
folders appear as the extension grows.
|
||||||
|
|
||||||
|
## Manifest Contract
|
||||||
|
|
||||||
|
`extension.json` must validate against:
|
||||||
|
|
||||||
|
```text
|
||||||
|
docs/schemas/extension-manifest.schema.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The key runtime fields are:
|
||||||
|
|
||||||
|
- `id`: must match the extension directory name.
|
||||||
|
- `extension_type`: one of the supported archetypes from the architecture
|
||||||
|
blueprint.
|
||||||
|
- `supported_frameworks`: framework IDs this extension can contribute evidence
|
||||||
|
for.
|
||||||
|
- `check_groups`: named groups that assessment profiles can select.
|
||||||
|
- `preflight_runner`: optional runner ID used before selected check groups.
|
||||||
|
- `runner_entrypoints`: concrete runner declarations.
|
||||||
|
- `certification_boundary`: explicit statement of what the extension does not
|
||||||
|
certify.
|
||||||
|
|
||||||
|
## Runner Entry Points
|
||||||
|
|
||||||
|
Runner entry points currently support these kinds:
|
||||||
|
|
||||||
|
- `python_module`: load a Python file from the extension directory and call a
|
||||||
|
function.
|
||||||
|
- `external`: declare an external harness that the baseline core cannot execute
|
||||||
|
yet.
|
||||||
|
- `command`: reserved for future command execution.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "cmis-browser-preflight",
|
||||||
|
"kind": "python_module",
|
||||||
|
"module_path": "src/open_cmis_tck/preflight.py",
|
||||||
|
"callable": "run",
|
||||||
|
"command": null,
|
||||||
|
"description": "Checks whether the CMIS Browser Binding endpoint is reachable."
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Python Runner Contract
|
||||||
|
|
||||||
|
A Python runner receives one context object and returns one result object.
|
||||||
|
|
||||||
|
```python
|
||||||
|
def run(context: dict) -> dict:
|
||||||
|
return {
|
||||||
|
"result": "pass",
|
||||||
|
"observations": ["Observed the expected condition."],
|
||||||
|
"facts": {"key": "value"},
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Context fields:
|
||||||
|
|
||||||
|
- `root`: repository root path as a string.
|
||||||
|
- `run_dir`: output run directory path as a string.
|
||||||
|
- `run_id`: current run ID.
|
||||||
|
- `plan`: full run plan snapshot.
|
||||||
|
- `step`: the step being executed.
|
||||||
|
- `target_profile`: target profile snapshot.
|
||||||
|
- `assessment_profile`: assessment profile snapshot.
|
||||||
|
- `extension_path`: extension directory path as a string.
|
||||||
|
- `runner`: manifest runner declaration.
|
||||||
|
|
||||||
|
Result fields:
|
||||||
|
|
||||||
|
- `result`: one of the guide-board evidence result statuses.
|
||||||
|
- `observations`: human-readable observations.
|
||||||
|
- `facts`: structured facts extracted by the runner.
|
||||||
|
- `artifact_refs`: references to raw artifacts written by the runner.
|
||||||
|
|
||||||
|
If a Python runner raises an exception, the core converts that failure into
|
||||||
|
`infrastructure_error` evidence so the assessment package remains complete.
|
||||||
|
|
||||||
|
## Result Statuses
|
||||||
|
|
||||||
|
Initial statuses:
|
||||||
|
|
||||||
|
- `pass`
|
||||||
|
- `fail`
|
||||||
|
- `warning`
|
||||||
|
- `manual`
|
||||||
|
- `not_applicable`
|
||||||
|
- `skipped`
|
||||||
|
- `expected_gap`
|
||||||
|
- `waiver_applied`
|
||||||
|
- `unsupported_by_design`
|
||||||
|
- `infrastructure_error`
|
||||||
|
- `blocked`
|
||||||
|
- `unknown`
|
||||||
|
|
||||||
|
## Current Extension Examples
|
||||||
|
|
||||||
|
- `sample-noop`: no runner, used to validate the core contracts.
|
||||||
|
- `open-cmis-tck`: provides a Python CMIS Browser Binding preflight runner and
|
||||||
|
declares the future external OpenCMIS TCK runner.
|
||||||
|
|
||||||
|
## Next SDK Steps
|
||||||
|
|
||||||
|
- Add command runner support with explicit allow/deny controls.
|
||||||
|
- Add artifact helper APIs for extension-generated raw files.
|
||||||
|
- Add normalizer and mapping plug-in contracts.
|
||||||
|
- Add extension-owned schema validation for domain-specific target profile
|
||||||
|
fields.
|
||||||
@@ -60,7 +60,25 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"preflight_runner": { "type": ["string", "null"] },
|
"preflight_runner": { "type": ["string", "null"] },
|
||||||
"runner_entrypoints": { "type": "array", "items": { "type": "string" } },
|
"runner_entrypoints": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": false,
|
||||||
|
"required": ["id", "kind"],
|
||||||
|
"properties": {
|
||||||
|
"id": { "type": "string" },
|
||||||
|
"kind": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": ["python_module", "command", "external"]
|
||||||
|
},
|
||||||
|
"module_path": { "type": ["string", "null"] },
|
||||||
|
"callable": { "type": ["string", "null"] },
|
||||||
|
"command": { "type": ["array", "null"], "items": { "type": "string" } },
|
||||||
|
"description": { "type": ["string", "null"] }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"normalizers": { "type": "array", "items": { "type": "string" } },
|
"normalizers": { "type": "array", "items": { "type": "string" } },
|
||||||
"mappings": { "type": "array", "items": { "type": "string" } },
|
"mappings": { "type": "array", "items": { "type": "string" } },
|
||||||
"report_fragments": { "type": "array", "items": { "type": "string" } },
|
"report_fragments": { "type": "array", "items": { "type": "string" } },
|
||||||
|
|||||||
@@ -12,7 +12,16 @@
|
|||||||
],
|
],
|
||||||
"check_groups": [],
|
"check_groups": [],
|
||||||
"preflight_runner": null,
|
"preflight_runner": null,
|
||||||
"runner_entrypoints": [],
|
"runner_entrypoints": [
|
||||||
|
{
|
||||||
|
"id": "replace-with-runner-id",
|
||||||
|
"kind": "external",
|
||||||
|
"module_path": null,
|
||||||
|
"callable": null,
|
||||||
|
"command": null,
|
||||||
|
"description": "Describe how this runner is provided."
|
||||||
|
}
|
||||||
|
],
|
||||||
"normalizers": [],
|
"normalizers": [],
|
||||||
"mappings": [],
|
"mappings": [],
|
||||||
"report_fragments": [],
|
"report_fragments": [],
|
||||||
|
|||||||
@@ -59,7 +59,22 @@
|
|||||||
],
|
],
|
||||||
"preflight_runner": "cmis-browser-preflight",
|
"preflight_runner": "cmis-browser-preflight",
|
||||||
"runner_entrypoints": [
|
"runner_entrypoints": [
|
||||||
"opencmis-tck"
|
{
|
||||||
|
"id": "cmis-browser-preflight",
|
||||||
|
"kind": "python_module",
|
||||||
|
"module_path": "src/open_cmis_tck/preflight.py",
|
||||||
|
"callable": "run",
|
||||||
|
"command": null,
|
||||||
|
"description": "Checks whether the configured CMIS Browser Binding endpoint is reachable and returns parseable repository metadata."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "opencmis-tck",
|
||||||
|
"kind": "external",
|
||||||
|
"module_path": null,
|
||||||
|
"callable": null,
|
||||||
|
"command": null,
|
||||||
|
"description": "Placeholder for the Java/Maven Apache Chemistry OpenCMIS TCK runner."
|
||||||
|
}
|
||||||
],
|
],
|
||||||
"normalizers": [
|
"normalizers": [
|
||||||
"opencmis-result-normalizer"
|
"opencmis-result-normalizer"
|
||||||
|
|||||||
161
extensions/open-cmis-tck/src/open_cmis_tck/preflight.py
Normal file
161
extensions/open-cmis-tck/src/open_cmis_tck/preflight.py
Normal file
@@ -0,0 +1,161 @@
|
|||||||
|
"""CMIS Browser Binding preflight runner."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from typing import Any
|
||||||
|
from urllib.error import HTTPError, URLError
|
||||||
|
from urllib.request import Request, urlopen
|
||||||
|
|
||||||
|
|
||||||
|
def run(context: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
target = context["target_profile"]
|
||||||
|
endpoint = _browser_endpoint(target)
|
||||||
|
if endpoint is None:
|
||||||
|
return {
|
||||||
|
"result": "fail",
|
||||||
|
"observations": [
|
||||||
|
"Target profile does not declare a CMIS Browser Binding endpoint."
|
||||||
|
],
|
||||||
|
"facts": {
|
||||||
|
"endpoint_found": False,
|
||||||
|
},
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
timeout = _timeout_seconds(context)
|
||||||
|
request = Request(
|
||||||
|
endpoint["url"],
|
||||||
|
headers={
|
||||||
|
"Accept": "application/json, */*;q=0.1",
|
||||||
|
"User-Agent": "guide-board-open-cmis-tck-preflight/0.1.0",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urlopen(request, timeout=timeout) as response:
|
||||||
|
status_code = response.status
|
||||||
|
content_type = response.headers.get("Content-Type", "")
|
||||||
|
body = response.read(1024 * 1024)
|
||||||
|
except HTTPError as exc:
|
||||||
|
return {
|
||||||
|
"result": "infrastructure_error",
|
||||||
|
"observations": [
|
||||||
|
f"CMIS Browser Binding endpoint returned HTTP {exc.code}."
|
||||||
|
],
|
||||||
|
"facts": {
|
||||||
|
"endpoint_found": True,
|
||||||
|
"url": endpoint["url"],
|
||||||
|
"http_status": exc.code,
|
||||||
|
},
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
except URLError as exc:
|
||||||
|
return {
|
||||||
|
"result": "infrastructure_error",
|
||||||
|
"observations": [
|
||||||
|
f"CMIS Browser Binding endpoint is not reachable: {exc.reason}."
|
||||||
|
],
|
||||||
|
"facts": {
|
||||||
|
"endpoint_found": True,
|
||||||
|
"url": endpoint["url"],
|
||||||
|
"error": str(exc.reason),
|
||||||
|
},
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
except TimeoutError:
|
||||||
|
return {
|
||||||
|
"result": "infrastructure_error",
|
||||||
|
"observations": [
|
||||||
|
f"CMIS Browser Binding endpoint did not respond within {timeout} seconds."
|
||||||
|
],
|
||||||
|
"facts": {
|
||||||
|
"endpoint_found": True,
|
||||||
|
"url": endpoint["url"],
|
||||||
|
"timeout_seconds": timeout,
|
||||||
|
},
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
facts: dict[str, Any] = {
|
||||||
|
"endpoint_found": True,
|
||||||
|
"url": endpoint["url"],
|
||||||
|
"binding": endpoint["binding"],
|
||||||
|
"http_status": status_code,
|
||||||
|
"content_type": content_type,
|
||||||
|
}
|
||||||
|
|
||||||
|
parsed = _parse_json(body)
|
||||||
|
if parsed is None:
|
||||||
|
facts["json_detected"] = False
|
||||||
|
return {
|
||||||
|
"result": "warning",
|
||||||
|
"observations": [
|
||||||
|
"CMIS Browser Binding endpoint is reachable but did not return parseable JSON."
|
||||||
|
],
|
||||||
|
"facts": facts,
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
facts["json_detected"] = True
|
||||||
|
facts.update(_repository_facts(parsed))
|
||||||
|
return {
|
||||||
|
"result": "pass",
|
||||||
|
"observations": [
|
||||||
|
"CMIS Browser Binding endpoint is reachable and returned parseable JSON."
|
||||||
|
],
|
||||||
|
"facts": facts,
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _browser_endpoint(target: dict[str, Any]) -> dict[str, Any] | None:
|
||||||
|
for endpoint in target.get("endpoints", []):
|
||||||
|
if endpoint.get("binding") == "cmis-browser":
|
||||||
|
return endpoint
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _timeout_seconds(context: dict[str, Any]) -> float:
|
||||||
|
runtime_policy = context["assessment_profile"].get("runtime_policy", {})
|
||||||
|
configured = runtime_policy.get("timeout_seconds", 5)
|
||||||
|
if not isinstance(configured, (int, float)):
|
||||||
|
return 5.0
|
||||||
|
return max(1.0, min(float(configured), 10.0))
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_json(body: bytes) -> Any:
|
||||||
|
try:
|
||||||
|
return json.loads(body.decode("utf-8"))
|
||||||
|
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _repository_facts(value: Any) -> dict[str, Any]:
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
return {"repository_shape": "unknown"}
|
||||||
|
|
||||||
|
if "repositoryId" in value:
|
||||||
|
return {
|
||||||
|
"repository_shape": "single-repository-info",
|
||||||
|
"repository_ids": [value["repositoryId"]],
|
||||||
|
"cmis_version_supported": value.get("cmisVersionSupported"),
|
||||||
|
"capabilities_present": isinstance(value.get("capabilities"), dict),
|
||||||
|
}
|
||||||
|
|
||||||
|
repository_ids = []
|
||||||
|
for key, child in value.items():
|
||||||
|
if isinstance(child, dict) and (
|
||||||
|
"repositoryId" in child or "repositoryName" in child
|
||||||
|
):
|
||||||
|
repository_ids.append(str(child.get("repositoryId", key)))
|
||||||
|
|
||||||
|
if repository_ids:
|
||||||
|
return {
|
||||||
|
"repository_shape": "repository-map",
|
||||||
|
"repository_ids": repository_ids,
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
"repository_shape": "object",
|
||||||
|
"top_level_keys": sorted(str(key) for key in value.keys())[:20],
|
||||||
|
}
|
||||||
@@ -91,7 +91,7 @@ Acceptance:
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: OPEN-CMIS-TCK-WP-0001-T003
|
id: OPEN-CMIS-TCK-WP-0001-T003
|
||||||
status: todo
|
status: in_progress
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "6d45885b-78a4-4e8b-8fcc-b8d6488e703b"
|
state_hub_task_id: "6d45885b-78a4-4e8b-8fcc-b8d6488e703b"
|
||||||
```
|
```
|
||||||
@@ -103,6 +103,13 @@ Acceptance:
|
|||||||
- Unsupported optional capabilities can be accepted as expected gaps.
|
- Unsupported optional capabilities can be accepted as expected gaps.
|
||||||
- Preflight output is captured as structured JSON.
|
- Preflight output is captured as structured JSON.
|
||||||
|
|
||||||
|
Progress:
|
||||||
|
|
||||||
|
- The first CMIS Browser Binding preflight runner checks endpoint reachability
|
||||||
|
and parseable JSON repository metadata through the guide-board runner bridge.
|
||||||
|
- Capability flag normalization remains to be expanded after a live target sample
|
||||||
|
is captured.
|
||||||
|
|
||||||
## D1.4 - OpenCMIS TCK Runner Wrapper
|
## D1.4 - OpenCMIS TCK Runner Wrapper
|
||||||
|
|
||||||
```task
|
```task
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ from typing import Any
|
|||||||
|
|
||||||
from guide_board.io import write_json
|
from guide_board.io import write_json
|
||||||
from guide_board.planning import build_run_plan
|
from guide_board.planning import build_run_plan
|
||||||
|
from guide_board.runners import run_step
|
||||||
from guide_board.schema import assert_valid
|
from guide_board.schema import assert_valid
|
||||||
|
|
||||||
|
|
||||||
@@ -23,7 +24,10 @@ def run_assessment(
|
|||||||
run_dir = output_dir or root / "runs" / run_id
|
run_dir = output_dir or root / "runs" / run_id
|
||||||
created_at = _now()
|
created_at = _now()
|
||||||
|
|
||||||
evidence = [_evidence_for_step(run_id, plan, step) for step in plan["ordered_steps"]]
|
evidence = [
|
||||||
|
_evidence_for_step(root, run_dir, run_id, plan, step)
|
||||||
|
for step in plan["ordered_steps"]
|
||||||
|
]
|
||||||
for item in evidence:
|
for item in evidence:
|
||||||
assert_valid(item, "evidence-item")
|
assert_valid(item, "evidence-item")
|
||||||
|
|
||||||
@@ -53,19 +57,16 @@ def run_assessment(
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def _evidence_for_step(run_id: str, plan: dict[str, Any], step: dict[str, Any]) -> dict[str, Any]:
|
def _evidence_for_step(
|
||||||
|
root: Path,
|
||||||
|
run_dir: Path,
|
||||||
|
run_id: str,
|
||||||
|
plan: dict[str, Any],
|
||||||
|
step: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
now = _now()
|
now = _now()
|
||||||
runner_ref = step.get("runner_ref")
|
runner_ref = step.get("runner_ref")
|
||||||
if runner_ref is None:
|
runner_result = run_step(root, run_dir, run_id, plan, step)
|
||||||
result = "manual" if step["kind"] == "check_group" else "skipped"
|
|
||||||
observations = [
|
|
||||||
"No runner is configured for this step in the baseline core."
|
|
||||||
]
|
|
||||||
else:
|
|
||||||
result = "blocked"
|
|
||||||
observations = [
|
|
||||||
f"Runner {runner_ref!r} is declared but not implemented by the baseline core."
|
|
||||||
]
|
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"id": f"evidence:{step['id']}",
|
"id": f"evidence:{step['id']}",
|
||||||
@@ -73,14 +74,15 @@ def _evidence_for_step(run_id: str, plan: dict[str, Any], step: dict[str, Any])
|
|||||||
"extension_id": step["extension_id"],
|
"extension_id": step["extension_id"],
|
||||||
"check_id": step["id"],
|
"check_id": step["id"],
|
||||||
"subject_ref": plan["target_profile_snapshot"]["id"],
|
"subject_ref": plan["target_profile_snapshot"]["id"],
|
||||||
"result": result,
|
"result": runner_result["result"],
|
||||||
"observations": observations,
|
"observations": runner_result["observations"],
|
||||||
"facts": {
|
"facts": {
|
||||||
"step_kind": step["kind"],
|
"step_kind": step["kind"],
|
||||||
"runner_ref": runner_ref,
|
"runner_ref": runner_ref,
|
||||||
|
**runner_result["facts"],
|
||||||
},
|
},
|
||||||
"requirement_refs": _requirement_refs(plan, step),
|
"requirement_refs": _requirement_refs(plan, step),
|
||||||
"artifact_refs": [],
|
"artifact_refs": runner_result["artifact_refs"],
|
||||||
"started_at": now,
|
"started_at": now,
|
||||||
"completed_at": now,
|
"completed_at": now,
|
||||||
}
|
}
|
||||||
@@ -95,25 +97,38 @@ def _requirement_refs(plan: dict[str, Any], step: dict[str, Any]) -> list[str]:
|
|||||||
def _findings_for_evidence(run_id: str, evidence: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
def _findings_for_evidence(run_id: str, evidence: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||||
findings: list[dict[str, Any]] = []
|
findings: list[dict[str, Any]] = []
|
||||||
for item in evidence:
|
for item in evidence:
|
||||||
if item["result"] != "blocked":
|
if item["result"] not in {"blocked", "fail", "infrastructure_error"}:
|
||||||
continue
|
continue
|
||||||
|
classification = {
|
||||||
|
"blocked": "runner_not_implemented",
|
||||||
|
"fail": "check_failed",
|
||||||
|
"infrastructure_error": "infrastructure_error",
|
||||||
|
}[item["result"]]
|
||||||
findings.append(
|
findings.append(
|
||||||
{
|
{
|
||||||
"id": f"finding:{item['check_id']}",
|
"id": f"finding:{item['check_id']}",
|
||||||
"run_id": run_id,
|
"run_id": run_id,
|
||||||
"status": "blocked",
|
"status": item["result"],
|
||||||
"severity": "info",
|
"severity": "info" if item["result"] == "blocked" else "medium",
|
||||||
"classification": "runner_not_implemented",
|
"classification": classification,
|
||||||
"requirement_refs": item["requirement_refs"],
|
"requirement_refs": item["requirement_refs"],
|
||||||
"evidence_refs": [item["id"]],
|
"evidence_refs": [item["id"]],
|
||||||
"expected": True,
|
"expected": item["result"] == "blocked",
|
||||||
"waiver_ref": None,
|
"waiver_ref": None,
|
||||||
"remediation": "Implement or configure the declared extension runner.",
|
"remediation": _remediation_for_result(item["result"]),
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
return findings
|
return findings
|
||||||
|
|
||||||
|
|
||||||
|
def _remediation_for_result(result: str) -> str:
|
||||||
|
if result == "blocked":
|
||||||
|
return "Implement or configure the declared extension runner."
|
||||||
|
if result == "infrastructure_error":
|
||||||
|
return "Fix the target, network, credentials, or harness runtime and rerun the assessment."
|
||||||
|
return "Review the failed check and target implementation."
|
||||||
|
|
||||||
|
|
||||||
def _assessment_package(
|
def _assessment_package(
|
||||||
run_id: str,
|
run_id: str,
|
||||||
plan: dict[str, Any],
|
plan: dict[str, Any],
|
||||||
@@ -198,6 +213,8 @@ def _markdown_report(run_metadata: dict[str, Any], package: dict[str, Any]) -> s
|
|||||||
def _run_status(evidence: list[dict[str, Any]]) -> str:
|
def _run_status(evidence: list[dict[str, Any]]) -> str:
|
||||||
if any(item["result"] == "fail" for item in evidence):
|
if any(item["result"] == "fail" for item in evidence):
|
||||||
return "failed"
|
return "failed"
|
||||||
|
if any(item["result"] == "infrastructure_error" for item in evidence):
|
||||||
|
return "infrastructure_error"
|
||||||
if any(item["result"] == "blocked" for item in evidence):
|
if any(item["result"] == "blocked" for item in evidence):
|
||||||
return "blocked"
|
return "blocked"
|
||||||
return "completed"
|
return "completed"
|
||||||
|
|||||||
162
src/guide_board/runners.py
Normal file
162
src/guide_board/runners.py
Normal file
@@ -0,0 +1,162 @@
|
|||||||
|
"""Runner bridge for extension-provided checks."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
from pathlib import Path
|
||||||
|
from types import ModuleType
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
from guide_board.errors import ValidationError
|
||||||
|
from guide_board.io import load_json
|
||||||
|
|
||||||
|
|
||||||
|
RunnerCallable = Callable[[dict[str, Any]], dict[str, Any]]
|
||||||
|
|
||||||
|
|
||||||
|
def run_step(
|
||||||
|
root: Path,
|
||||||
|
run_dir: Path,
|
||||||
|
run_id: str,
|
||||||
|
plan: dict[str, Any],
|
||||||
|
step: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
runner_ref = step.get("runner_ref")
|
||||||
|
if runner_ref is None:
|
||||||
|
return _no_runner_result(step)
|
||||||
|
|
||||||
|
extension = _extension_snapshot(plan, step["extension_id"])
|
||||||
|
extension_path = root / extension["path"]
|
||||||
|
manifest = load_json(extension_path / "extension.json")
|
||||||
|
entrypoint = _runner_entrypoint(manifest, runner_ref)
|
||||||
|
if entrypoint["kind"] == "python_module":
|
||||||
|
return _run_python_module(root, run_dir, run_id, plan, step, extension_path, entrypoint)
|
||||||
|
if entrypoint["kind"] == "external":
|
||||||
|
return {
|
||||||
|
"result": "blocked",
|
||||||
|
"observations": [
|
||||||
|
f"Runner {runner_ref!r} is declared as an external runner and is not implemented by the core."
|
||||||
|
],
|
||||||
|
"facts": {
|
||||||
|
"runner_ref": runner_ref,
|
||||||
|
"runner_kind": "external",
|
||||||
|
},
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
if entrypoint["kind"] == "command":
|
||||||
|
return {
|
||||||
|
"result": "blocked",
|
||||||
|
"observations": [
|
||||||
|
f"Runner {runner_ref!r} is declared as a command runner; command execution is not enabled yet."
|
||||||
|
],
|
||||||
|
"facts": {
|
||||||
|
"runner_ref": runner_ref,
|
||||||
|
"runner_kind": "command",
|
||||||
|
},
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
raise ValidationError(f"{runner_ref}: unsupported runner kind {entrypoint['kind']!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def _no_runner_result(step: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
result = "manual" if step["kind"] == "check_group" else "skipped"
|
||||||
|
return {
|
||||||
|
"result": result,
|
||||||
|
"observations": [
|
||||||
|
"No runner is configured for this step in the baseline core."
|
||||||
|
],
|
||||||
|
"facts": {
|
||||||
|
"runner_ref": None,
|
||||||
|
"runner_kind": None,
|
||||||
|
},
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _run_python_module(
|
||||||
|
root: Path,
|
||||||
|
run_dir: Path,
|
||||||
|
run_id: str,
|
||||||
|
plan: dict[str, Any],
|
||||||
|
step: dict[str, Any],
|
||||||
|
extension_path: Path,
|
||||||
|
entrypoint: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
module_path = entrypoint.get("module_path")
|
||||||
|
callable_name = entrypoint.get("callable")
|
||||||
|
if not module_path or not callable_name:
|
||||||
|
raise ValidationError(f"{entrypoint['id']}: python_module runners need module_path and callable")
|
||||||
|
|
||||||
|
module_file = (extension_path / module_path).resolve()
|
||||||
|
try:
|
||||||
|
module_file.relative_to(extension_path.resolve())
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ValidationError(
|
||||||
|
f"{entrypoint['id']}: module_path must stay inside the extension directory"
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
module = _load_module(module_file, entrypoint["id"])
|
||||||
|
runner = getattr(module, callable_name, None)
|
||||||
|
if not callable(runner):
|
||||||
|
raise ValidationError(f"{entrypoint['id']}: callable {callable_name!r} was not found")
|
||||||
|
|
||||||
|
context = {
|
||||||
|
"root": str(root),
|
||||||
|
"run_dir": str(run_dir),
|
||||||
|
"run_id": run_id,
|
||||||
|
"plan": plan,
|
||||||
|
"step": step,
|
||||||
|
"target_profile": plan["target_profile_snapshot"],
|
||||||
|
"assessment_profile": plan["assessment_profile_snapshot"],
|
||||||
|
"extension_path": str(extension_path),
|
||||||
|
"runner": entrypoint,
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
result = runner(context)
|
||||||
|
except Exception as exc: # noqa: BLE001 - extension failures become evidence.
|
||||||
|
return {
|
||||||
|
"result": "infrastructure_error",
|
||||||
|
"observations": [
|
||||||
|
f"Runner {entrypoint['id']!r} failed before producing evidence: {exc}"
|
||||||
|
],
|
||||||
|
"facts": {
|
||||||
|
"runner_ref": entrypoint["id"],
|
||||||
|
"runner_kind": "python_module",
|
||||||
|
"error_type": type(exc).__name__,
|
||||||
|
},
|
||||||
|
"artifact_refs": [],
|
||||||
|
}
|
||||||
|
if not isinstance(result, dict):
|
||||||
|
raise ValidationError(f"{entrypoint['id']}: runner must return an object")
|
||||||
|
return {
|
||||||
|
"result": result.get("result", "unknown"),
|
||||||
|
"observations": result.get("observations", []),
|
||||||
|
"facts": result.get("facts", {}),
|
||||||
|
"artifact_refs": result.get("artifact_refs", []),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _load_module(path: Path, runner_id: str) -> ModuleType:
|
||||||
|
if not path.exists():
|
||||||
|
raise ValidationError(f"{runner_id}: module not found: {path}")
|
||||||
|
module_name = f"_guide_board_runner_{runner_id.replace('-', '_')}"
|
||||||
|
spec = importlib.util.spec_from_file_location(module_name, path)
|
||||||
|
if spec is None or spec.loader is None:
|
||||||
|
raise ValidationError(f"{runner_id}: unable to load module from {path}")
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
def _extension_snapshot(plan: dict[str, Any], extension_id: str) -> dict[str, Any]:
|
||||||
|
for extension in plan["extension_snapshots"]:
|
||||||
|
if extension["id"] == extension_id:
|
||||||
|
return extension
|
||||||
|
raise ValidationError(f"step references unknown extension {extension_id!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def _runner_entrypoint(manifest: dict[str, Any], runner_ref: str) -> dict[str, Any]:
|
||||||
|
for entrypoint in manifest.get("runner_entrypoints", []):
|
||||||
|
if entrypoint["id"] == runner_ref:
|
||||||
|
return entrypoint
|
||||||
|
raise ValidationError(f"{manifest['id']}: runner {runner_ref!r} is not declared")
|
||||||
16
src/guide_board/sdk.py
Normal file
16
src/guide_board/sdk.py
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
"""Public helper types for extension runners.
|
||||||
|
|
||||||
|
Extension Python runners are called with one dictionary context and should return
|
||||||
|
one dictionary shaped like `RunnerResult`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, TypedDict
|
||||||
|
|
||||||
|
|
||||||
|
class RunnerResult(TypedDict, total=False):
|
||||||
|
result: str
|
||||||
|
observations: list[str]
|
||||||
|
facts: dict[str, Any]
|
||||||
|
artifact_refs: list[str]
|
||||||
@@ -1,6 +1,9 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import unittest
|
import unittest
|
||||||
|
import json
|
||||||
|
import threading
|
||||||
|
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||||
from tempfile import TemporaryDirectory
|
from tempfile import TemporaryDirectory
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
@@ -79,6 +82,111 @@ class CoreArchitectureTests(unittest.TestCase):
|
|||||||
self.assertTrue((run_dir / "reports" / "assessment-package.json").exists())
|
self.assertTrue((run_dir / "reports" / "assessment-package.json").exists())
|
||||||
self.assertTrue((run_dir / "reports" / "report.md").exists())
|
self.assertTrue((run_dir / "reports" / "report.md").exists())
|
||||||
|
|
||||||
|
def test_runs_cmis_preflight_against_local_endpoint(self) -> None:
|
||||||
|
server = HTTPServer(("127.0.0.1", 0), _CmisHandler)
|
||||||
|
thread = threading.Thread(target=server.serve_forever)
|
||||||
|
thread.daemon = True
|
||||||
|
thread.start()
|
||||||
|
try:
|
||||||
|
with TemporaryDirectory() as temporary_directory:
|
||||||
|
temp_root = Path(temporary_directory)
|
||||||
|
target_path = temp_root / "target.json"
|
||||||
|
assessment_path = temp_root / "assessment.json"
|
||||||
|
target_path.write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"id": "local-cmis-test",
|
||||||
|
"subject_type": "cmis-browser-binding-endpoint",
|
||||||
|
"subject_name": "Local CMIS Test",
|
||||||
|
"environment": "test",
|
||||||
|
"scope": ["preflight"],
|
||||||
|
"endpoints": [
|
||||||
|
{
|
||||||
|
"id": "browser-binding",
|
||||||
|
"url": f"http://127.0.0.1:{server.server_port}/cmis/browser",
|
||||||
|
"binding": "cmis-browser",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"artifacts": [],
|
||||||
|
"credentials_ref": None,
|
||||||
|
"declared_capabilities": ["cmis.repository-info"],
|
||||||
|
"known_gaps": [],
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
assessment_path.write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"id": "local-cmis-preflight",
|
||||||
|
"framework_refs": ["cmis.browser-binding.compatibility.v1"],
|
||||||
|
"extension_refs": ["open-cmis-tck"],
|
||||||
|
"target_profile_ref": "local-cmis-test",
|
||||||
|
"selected_check_groups": {"open-cmis-tck": []},
|
||||||
|
"expectations_ref": None,
|
||||||
|
"waivers_ref": None,
|
||||||
|
"output_policy": {
|
||||||
|
"report_formats": ["json", "markdown"],
|
||||||
|
"artifact_retention": "summary-only",
|
||||||
|
},
|
||||||
|
"retention_policy": {
|
||||||
|
"summary_days": 365,
|
||||||
|
"raw_artifact_days": 0,
|
||||||
|
},
|
||||||
|
"runtime_policy": {
|
||||||
|
"offline": False,
|
||||||
|
"timeout_seconds": 2,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
result = run_assessment(
|
||||||
|
ROOT,
|
||||||
|
target_path,
|
||||||
|
assessment_path,
|
||||||
|
temp_root / "run",
|
||||||
|
)
|
||||||
|
evidence = json.loads(
|
||||||
|
(Path(result["run_dir"]) / "normalized" / "evidence.json").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(result["status"], "completed")
|
||||||
|
self.assertEqual(evidence["evidence"][0]["result"], "pass")
|
||||||
|
self.assertEqual(
|
||||||
|
evidence["evidence"][0]["facts"]["repository_ids"],
|
||||||
|
["local-test-repository"],
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
server.shutdown()
|
||||||
|
thread.join(timeout=5)
|
||||||
|
server.server_close()
|
||||||
|
|
||||||
|
|
||||||
|
class _CmisHandler(BaseHTTPRequestHandler):
|
||||||
|
def do_GET(self) -> None:
|
||||||
|
body = json.dumps(
|
||||||
|
{
|
||||||
|
"local-test-repository": {
|
||||||
|
"repositoryId": "local-test-repository",
|
||||||
|
"repositoryName": "Local Test Repository",
|
||||||
|
"cmisVersionSupported": "1.1",
|
||||||
|
"capabilities": {},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
).encode("utf-8")
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", "application/json")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def log_message(self, format: str, *args: object) -> None:
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
@@ -189,7 +189,7 @@ Acceptance:
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: GUIDE-BOARD-WP-0001-T006
|
id: GUIDE-BOARD-WP-0001-T006
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "3c757929-a5e4-4c11-bbf1-6d7f26def93e"
|
state_hub_task_id: "3c757929-a5e4-4c11-bbf1-6d7f26def93e"
|
||||||
```
|
```
|
||||||
@@ -201,12 +201,13 @@ Acceptance:
|
|||||||
- Provide a minimal extension template.
|
- Provide a minimal extension template.
|
||||||
- Extension ownership boundaries make later extraction to a separate repository
|
- Extension ownership boundaries make later extraction to a separate repository
|
||||||
straightforward.
|
straightforward.
|
||||||
|
- Python module runner contracts are documented in `docs/EXTENSION-SDK.md`.
|
||||||
|
|
||||||
## D1.8 - CMIS Seed Extension Integration
|
## D1.8 - CMIS Seed Extension Integration
|
||||||
|
|
||||||
```task
|
```task
|
||||||
id: GUIDE-BOARD-WP-0001-T007
|
id: GUIDE-BOARD-WP-0001-T007
|
||||||
status: todo
|
status: in_progress
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "455f92b0-1d2b-43d0-aa61-464d9dc83a62"
|
state_hub_task_id: "455f92b0-1d2b-43d0-aa61-464d9dc83a62"
|
||||||
```
|
```
|
||||||
@@ -218,6 +219,14 @@ Acceptance:
|
|||||||
- CMIS output normalizes into the same evidence model used by other extensions.
|
- CMIS output normalizes into the same evidence model used by other extensions.
|
||||||
- CMIS capability mappings are extension-owned.
|
- CMIS capability mappings are extension-owned.
|
||||||
|
|
||||||
|
Progress:
|
||||||
|
|
||||||
|
- `open-cmis-tck` declares runner entry points through `extension.json`.
|
||||||
|
- The CMIS Browser Binding preflight runner executes through the generic runner
|
||||||
|
bridge and produces normalized evidence.
|
||||||
|
- The OpenCMIS Java/Maven TCK runner remains external and blocked until the
|
||||||
|
extension harness wrapper is implemented.
|
||||||
|
|
||||||
## D1.9 - Containerized Execution Design
|
## D1.9 - Containerized Execution Design
|
||||||
|
|
||||||
```task
|
```task
|
||||||
|
|||||||
Reference in New Issue
Block a user