diff --git a/.custodian-brief.md b/.custodian-brief.md index 0e61474..fa26666 100644 --- a/.custodian-brief.md +++ b/.custodian-brief.md @@ -2,16 +2,12 @@ # Custodian Brief — guide-board **Domain:** markitect -**Last synced:** 2026-05-15 12:43 UTC +**Last synced:** 2026-05-15 12:56 UTC **State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)* ## Active Workstreams -### Assessment Operations Baseline -Progress: 5/6 done | workstream_id: `fc5b1573-91b2-4a19-b6a9-dd4d17057d9b` - -**Open tasks:** -- · D2.6 - External Extension Acceptance Path `65fbf1df` +*(none — repo may need first-session setup)* --- ## MCP Orientation (when available) diff --git a/README.md b/README.md index 941c2c3..c0f5aaf 100644 --- a/README.md +++ b/README.md @@ -54,6 +54,7 @@ See: - [docs/CANDIDATE-HANDOFF.md](docs/CANDIDATE-HANDOFF.md) - [docs/COMPLIANCE-EVIDENCE-PACKS.md](docs/COMPLIANCE-EVIDENCE-PACKS.md) - [docs/CONTAINER.md](docs/CONTAINER.md) +- [docs/EXTERNAL-EXTENSION-ACCEPTANCE.md](docs/EXTERNAL-EXTENSION-ACCEPTANCE.md) - [docs/EXTENSION-SDK.md](docs/EXTENSION-SDK.md) - [docs/LOCAL-SERVICE-API.md](docs/LOCAL-SERVICE-API.md) - [docs/SERVICE-JOB-DURABILITY.md](docs/SERVICE-JOB-DURABILITY.md) diff --git a/docs/ASSESSMENT-OPERATIONS.md b/docs/ASSESSMENT-OPERATIONS.md index bb3d059..052c07a 100644 --- a/docs/ASSESSMENT-OPERATIONS.md +++ b/docs/ASSESSMENT-OPERATIONS.md @@ -62,6 +62,9 @@ PYTHONPATH=src python3 -m guide_board \ The same extension roots can be provided through `GUIDE_BOARD_EXTENSION_PATHS` when a wrapper script or container entrypoint should keep commands shorter. +For the repeatable external extension acceptance path, including validation, +planning, live execution, and retained result review, see +`docs/EXTERNAL-EXTENSION-ACCEPTANCE.md`. ## CLI Results diff --git a/docs/EXTERNAL-EXTENSION-ACCEPTANCE.md b/docs/EXTERNAL-EXTENSION-ACCEPTANCE.md new file mode 100644 index 0000000..2a810e4 --- /dev/null +++ b/docs/EXTERNAL-EXTENSION-ACCEPTANCE.md @@ -0,0 +1,144 @@ +# External Extension Acceptance + +Status: draft +Created: 2026-05-15 + +## Purpose + +This document defines the repeatable acceptance path for a guide-board +extension that lives outside the core repository. It proves that guide-board can +discover the extension, validate candidate profiles, build a traceable run plan, +execute the selected checks, and review retained results without moving +domain-specific harness logic into the core. + +`open-cmis-tck` is the first concrete example. CMIS-specific profiles, runners, +runtime dependencies, and harness behavior remain owned by that extension +repository. + +## Acceptance Stages + +Run these stages from the guide-board repository. + +1. Confirm the extension repository shape: + +```sh +test -f ../open-cmis-tck/extension.json +``` + +2. Validate extension discovery and manifest contracts: + +```sh +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + extensions validate +``` + +3. Validate the target and assessment profiles: + +```sh +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + profile validate-target \ + ../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json + +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + profile validate-assessment \ + ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json +``` + +4. Generate and retain a run plan: + +```sh +mkdir -p /tmp/guide-board-external-extension-acceptance +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + plan \ + --target ../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json \ + --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \ + --output /tmp/guide-board-external-extension-acceptance/plan.json +``` + +5. Execute a live run when the candidate endpoint and extension runtime + dependencies are available: + +```sh +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + run \ + --target ../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json \ + --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \ + --output-dir /tmp/guide-board-external-extension-acceptance/open-cmis-tck-baseline +``` + +6. Review retained results: + +```sh +PYTHONPATH=src python3 -m guide_board runs report \ + --runs-dir /tmp/guide-board-external-extension-acceptance \ + --target kontextual-cmis-compat \ + --assessment cmis-browser-baseline +``` + +The run directory should contain: + +```text +run.json +plan.json +retention-summary.json +normalized/evidence.json +normalized/findings.json +normalized/mappings.json +reports/assessment-package.json +reports/report.md +artifacts/ +``` + +## Scripted Acceptance + +The scripted path keeps the same commands in one place: + +```sh +scripts/external_extension_acceptance.sh plan +``` + +The default `plan` mode validates the extension and profiles, then writes a run +plan under `/tmp/guide-board-external-extension-acceptance`. It does not contact +the candidate system. + +Use `run` mode for the live acceptance after the candidate endpoint and harness +dependencies are ready: + +```sh +scripts/external_extension_acceptance.sh run +``` + +Override the extension, profiles, or output location when accepting another +extension: + +```sh +GUIDE_BOARD_ACCEPT_EXTENSION_DIR=../example-extension \ +GUIDE_BOARD_ACCEPT_TARGET=../example-extension/profiles/targets/example.json \ +GUIDE_BOARD_ACCEPT_ASSESSMENT=../example-extension/profiles/assessments/example.json \ +GUIDE_BOARD_ACCEPT_OUTPUT_DIR=/tmp/guide-board-example-extension \ +scripts/external_extension_acceptance.sh plan +``` + +## Ownership Boundary + +Guide-board owns: + +- extension discovery, +- extension manifest validation, +- target and assessment profile validation, +- run plan construction, +- run artifact layout, +- retained run summary and report lookup. + +The external extension repository owns: + +- domain-specific target and assessment profiles, +- harness adapters and runtime dependencies, +- credentials or restricted asset instructions, +- check group behavior, +- interpretation of domain-specific evidence. diff --git a/scripts/external_extension_acceptance.sh b/scripts/external_extension_acceptance.sh new file mode 100755 index 0000000..e3200a3 --- /dev/null +++ b/scripts/external_extension_acceptance.sh @@ -0,0 +1,106 @@ +#!/usr/bin/env sh +set -eu + +ROOT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)" +MODE="${1:-plan}" +EXTENSION_DIR="${GUIDE_BOARD_ACCEPT_EXTENSION_DIR:-../open-cmis-tck}" +TARGET="${GUIDE_BOARD_ACCEPT_TARGET:-../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json}" +ASSESSMENT="${GUIDE_BOARD_ACCEPT_ASSESSMENT:-../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json}" +OUTPUT_DIR="${GUIDE_BOARD_ACCEPT_OUTPUT_DIR:-${TMPDIR:-/tmp}/guide-board-external-extension-acceptance}" +PLAN_OUTPUT="${GUIDE_BOARD_ACCEPT_PLAN_OUTPUT:-$OUTPUT_DIR/plan.json}" +RUN_DIR="${GUIDE_BOARD_ACCEPT_RUN_DIR:-$OUTPUT_DIR/open-cmis-tck-baseline}" +PYTHON="${PYTHON:-python3}" + +case "$MODE" in + plan|run) + ;; + *) + echo "Usage: $0 [plan|run]" >&2 + exit 64 + ;; +esac + +cd "$ROOT_DIR" +export PYTHONPATH="$ROOT_DIR/src${PYTHONPATH:+:$PYTHONPATH}" + +if [ ! -f "$EXTENSION_DIR/extension.json" ]; then + echo "ERROR: extension manifest not found at $EXTENSION_DIR/extension.json" >&2 + exit 66 +fi + +mkdir -p "$OUTPUT_DIR" + +echo "==> Listing discovered extensions" +"$PYTHON" -m guide_board \ + --root "$ROOT_DIR" \ + --extension-dir "$EXTENSION_DIR" \ + extensions list + +echo "==> Validating extension manifests" +"$PYTHON" -m guide_board \ + --root "$ROOT_DIR" \ + --extension-dir "$EXTENSION_DIR" \ + extensions validate + +echo "==> Validating target profile" +"$PYTHON" -m guide_board \ + --root "$ROOT_DIR" \ + --extension-dir "$EXTENSION_DIR" \ + profile validate-target "$TARGET" + +echo "==> Validating assessment profile" +"$PYTHON" -m guide_board \ + --root "$ROOT_DIR" \ + --extension-dir "$EXTENSION_DIR" \ + profile validate-assessment "$ASSESSMENT" + +echo "==> Generating run plan" +"$PYTHON" -m guide_board \ + --root "$ROOT_DIR" \ + --extension-dir "$EXTENSION_DIR" \ + plan \ + --target "$TARGET" \ + --assessment "$ASSESSMENT" \ + --output "$PLAN_OUTPUT" + +if [ "$MODE" = "plan" ]; then + echo "External extension acceptance plan passed." + echo "Plan output: $PLAN_OUTPUT" + echo "Run live acceptance with: $0 run" + exit 0 +fi + +echo "==> Running live assessment" +"$PYTHON" -m guide_board \ + --root "$ROOT_DIR" \ + --extension-dir "$EXTENSION_DIR" \ + run \ + --target "$TARGET" \ + --assessment "$ASSESSMENT" \ + --output-dir "$RUN_DIR" + +echo "==> Verifying retained run artifacts" +for path in \ + "$RUN_DIR/run.json" \ + "$RUN_DIR/plan.json" \ + "$RUN_DIR/retention-summary.json" \ + "$RUN_DIR/normalized/evidence.json" \ + "$RUN_DIR/normalized/findings.json" \ + "$RUN_DIR/normalized/mappings.json" \ + "$RUN_DIR/reports/assessment-package.json" \ + "$RUN_DIR/reports/report.md" +do + if [ ! -f "$path" ]; then + echo "ERROR: expected artifact missing: $path" >&2 + exit 1 + fi +done + +echo "==> Reviewing retained report paths" +"$PYTHON" -m guide_board \ + --root "$ROOT_DIR" \ + runs report \ + --runs-dir "$OUTPUT_DIR" + +echo "External extension live acceptance passed." +echo "Run artifacts: $RUN_DIR" diff --git a/workplans/GUIDE-BOARD-WP-0002-assessment-operations-baseline.md b/workplans/GUIDE-BOARD-WP-0002-assessment-operations-baseline.md index 170a497..9c3c130 100644 --- a/workplans/GUIDE-BOARD-WP-0002-assessment-operations-baseline.md +++ b/workplans/GUIDE-BOARD-WP-0002-assessment-operations-baseline.md @@ -4,7 +4,7 @@ type: workplan title: "Assessment Operations Baseline" repo: guide-board domain: markitect -status: active +status: completed owner: codex planning_priority: high planning_order: 2 @@ -174,7 +174,7 @@ Progress: ```task id: GUIDE-BOARD-WP-0002-T006 -status: todo +status: done priority: high state_hub_task_id: "65fbf1df-caef-40f6-abee-8308daf27fbc" ``` @@ -187,6 +187,13 @@ Acceptance: - Cover extension validation, target/assessment profile validation, plan generation, run execution, and result review. +Progress: + +- Added `docs/EXTERNAL-EXTENSION-ACCEPTANCE.md`. +- Added `scripts/external_extension_acceptance.sh`. +- Verified `open-cmis-tck` extension validation, profile validation, and plan + generation in scripted plan mode. + ## Definition Of Done - The repo has a clear assessment operations guide.