generated from coulomb/repo-seed
Add external extension acceptance path
This commit is contained in:
@@ -2,16 +2,12 @@
|
|||||||
# Custodian Brief — guide-board
|
# Custodian Brief — guide-board
|
||||||
|
|
||||||
**Domain:** markitect
|
**Domain:** markitect
|
||||||
**Last synced:** 2026-05-15 12:43 UTC
|
**Last synced:** 2026-05-15 12:56 UTC
|
||||||
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
||||||
|
|
||||||
## Active Workstreams
|
## Active Workstreams
|
||||||
|
|
||||||
### Assessment Operations Baseline
|
*(none — repo may need first-session setup)*
|
||||||
Progress: 5/6 done | workstream_id: `fc5b1573-91b2-4a19-b6a9-dd4d17057d9b`
|
|
||||||
|
|
||||||
**Open tasks:**
|
|
||||||
- · D2.6 - External Extension Acceptance Path `65fbf1df`
|
|
||||||
|
|
||||||
---
|
---
|
||||||
## MCP Orientation (when available)
|
## MCP Orientation (when available)
|
||||||
|
|||||||
@@ -54,6 +54,7 @@ See:
|
|||||||
- [docs/CANDIDATE-HANDOFF.md](docs/CANDIDATE-HANDOFF.md)
|
- [docs/CANDIDATE-HANDOFF.md](docs/CANDIDATE-HANDOFF.md)
|
||||||
- [docs/COMPLIANCE-EVIDENCE-PACKS.md](docs/COMPLIANCE-EVIDENCE-PACKS.md)
|
- [docs/COMPLIANCE-EVIDENCE-PACKS.md](docs/COMPLIANCE-EVIDENCE-PACKS.md)
|
||||||
- [docs/CONTAINER.md](docs/CONTAINER.md)
|
- [docs/CONTAINER.md](docs/CONTAINER.md)
|
||||||
|
- [docs/EXTERNAL-EXTENSION-ACCEPTANCE.md](docs/EXTERNAL-EXTENSION-ACCEPTANCE.md)
|
||||||
- [docs/EXTENSION-SDK.md](docs/EXTENSION-SDK.md)
|
- [docs/EXTENSION-SDK.md](docs/EXTENSION-SDK.md)
|
||||||
- [docs/LOCAL-SERVICE-API.md](docs/LOCAL-SERVICE-API.md)
|
- [docs/LOCAL-SERVICE-API.md](docs/LOCAL-SERVICE-API.md)
|
||||||
- [docs/SERVICE-JOB-DURABILITY.md](docs/SERVICE-JOB-DURABILITY.md)
|
- [docs/SERVICE-JOB-DURABILITY.md](docs/SERVICE-JOB-DURABILITY.md)
|
||||||
|
|||||||
@@ -62,6 +62,9 @@ PYTHONPATH=src python3 -m guide_board \
|
|||||||
|
|
||||||
The same extension roots can be provided through `GUIDE_BOARD_EXTENSION_PATHS`
|
The same extension roots can be provided through `GUIDE_BOARD_EXTENSION_PATHS`
|
||||||
when a wrapper script or container entrypoint should keep commands shorter.
|
when a wrapper script or container entrypoint should keep commands shorter.
|
||||||
|
For the repeatable external extension acceptance path, including validation,
|
||||||
|
planning, live execution, and retained result review, see
|
||||||
|
`docs/EXTERNAL-EXTENSION-ACCEPTANCE.md`.
|
||||||
|
|
||||||
## CLI Results
|
## CLI Results
|
||||||
|
|
||||||
|
|||||||
144
docs/EXTERNAL-EXTENSION-ACCEPTANCE.md
Normal file
144
docs/EXTERNAL-EXTENSION-ACCEPTANCE.md
Normal file
@@ -0,0 +1,144 @@
|
|||||||
|
# External Extension Acceptance
|
||||||
|
|
||||||
|
Status: draft
|
||||||
|
Created: 2026-05-15
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This document defines the repeatable acceptance path for a guide-board
|
||||||
|
extension that lives outside the core repository. It proves that guide-board can
|
||||||
|
discover the extension, validate candidate profiles, build a traceable run plan,
|
||||||
|
execute the selected checks, and review retained results without moving
|
||||||
|
domain-specific harness logic into the core.
|
||||||
|
|
||||||
|
`open-cmis-tck` is the first concrete example. CMIS-specific profiles, runners,
|
||||||
|
runtime dependencies, and harness behavior remain owned by that extension
|
||||||
|
repository.
|
||||||
|
|
||||||
|
## Acceptance Stages
|
||||||
|
|
||||||
|
Run these stages from the guide-board repository.
|
||||||
|
|
||||||
|
1. Confirm the extension repository shape:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
test -f ../open-cmis-tck/extension.json
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Validate extension discovery and manifest contracts:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
PYTHONPATH=src python3 -m guide_board \
|
||||||
|
--extension-dir ../open-cmis-tck \
|
||||||
|
extensions validate
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Validate the target and assessment profiles:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
PYTHONPATH=src python3 -m guide_board \
|
||||||
|
--extension-dir ../open-cmis-tck \
|
||||||
|
profile validate-target \
|
||||||
|
../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json
|
||||||
|
|
||||||
|
PYTHONPATH=src python3 -m guide_board \
|
||||||
|
--extension-dir ../open-cmis-tck \
|
||||||
|
profile validate-assessment \
|
||||||
|
../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Generate and retain a run plan:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mkdir -p /tmp/guide-board-external-extension-acceptance
|
||||||
|
PYTHONPATH=src python3 -m guide_board \
|
||||||
|
--extension-dir ../open-cmis-tck \
|
||||||
|
plan \
|
||||||
|
--target ../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json \
|
||||||
|
--assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \
|
||||||
|
--output /tmp/guide-board-external-extension-acceptance/plan.json
|
||||||
|
```
|
||||||
|
|
||||||
|
5. Execute a live run when the candidate endpoint and extension runtime
|
||||||
|
dependencies are available:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
PYTHONPATH=src python3 -m guide_board \
|
||||||
|
--extension-dir ../open-cmis-tck \
|
||||||
|
run \
|
||||||
|
--target ../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json \
|
||||||
|
--assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \
|
||||||
|
--output-dir /tmp/guide-board-external-extension-acceptance/open-cmis-tck-baseline
|
||||||
|
```
|
||||||
|
|
||||||
|
6. Review retained results:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
PYTHONPATH=src python3 -m guide_board runs report \
|
||||||
|
--runs-dir /tmp/guide-board-external-extension-acceptance \
|
||||||
|
--target kontextual-cmis-compat \
|
||||||
|
--assessment cmis-browser-baseline
|
||||||
|
```
|
||||||
|
|
||||||
|
The run directory should contain:
|
||||||
|
|
||||||
|
```text
|
||||||
|
run.json
|
||||||
|
plan.json
|
||||||
|
retention-summary.json
|
||||||
|
normalized/evidence.json
|
||||||
|
normalized/findings.json
|
||||||
|
normalized/mappings.json
|
||||||
|
reports/assessment-package.json
|
||||||
|
reports/report.md
|
||||||
|
artifacts/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Scripted Acceptance
|
||||||
|
|
||||||
|
The scripted path keeps the same commands in one place:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
scripts/external_extension_acceptance.sh plan
|
||||||
|
```
|
||||||
|
|
||||||
|
The default `plan` mode validates the extension and profiles, then writes a run
|
||||||
|
plan under `/tmp/guide-board-external-extension-acceptance`. It does not contact
|
||||||
|
the candidate system.
|
||||||
|
|
||||||
|
Use `run` mode for the live acceptance after the candidate endpoint and harness
|
||||||
|
dependencies are ready:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
scripts/external_extension_acceptance.sh run
|
||||||
|
```
|
||||||
|
|
||||||
|
Override the extension, profiles, or output location when accepting another
|
||||||
|
extension:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GUIDE_BOARD_ACCEPT_EXTENSION_DIR=../example-extension \
|
||||||
|
GUIDE_BOARD_ACCEPT_TARGET=../example-extension/profiles/targets/example.json \
|
||||||
|
GUIDE_BOARD_ACCEPT_ASSESSMENT=../example-extension/profiles/assessments/example.json \
|
||||||
|
GUIDE_BOARD_ACCEPT_OUTPUT_DIR=/tmp/guide-board-example-extension \
|
||||||
|
scripts/external_extension_acceptance.sh plan
|
||||||
|
```
|
||||||
|
|
||||||
|
## Ownership Boundary
|
||||||
|
|
||||||
|
Guide-board owns:
|
||||||
|
|
||||||
|
- extension discovery,
|
||||||
|
- extension manifest validation,
|
||||||
|
- target and assessment profile validation,
|
||||||
|
- run plan construction,
|
||||||
|
- run artifact layout,
|
||||||
|
- retained run summary and report lookup.
|
||||||
|
|
||||||
|
The external extension repository owns:
|
||||||
|
|
||||||
|
- domain-specific target and assessment profiles,
|
||||||
|
- harness adapters and runtime dependencies,
|
||||||
|
- credentials or restricted asset instructions,
|
||||||
|
- check group behavior,
|
||||||
|
- interpretation of domain-specific evidence.
|
||||||
106
scripts/external_extension_acceptance.sh
Executable file
106
scripts/external_extension_acceptance.sh
Executable file
@@ -0,0 +1,106 @@
|
|||||||
|
#!/usr/bin/env sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)"
|
||||||
|
MODE="${1:-plan}"
|
||||||
|
EXTENSION_DIR="${GUIDE_BOARD_ACCEPT_EXTENSION_DIR:-../open-cmis-tck}"
|
||||||
|
TARGET="${GUIDE_BOARD_ACCEPT_TARGET:-../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json}"
|
||||||
|
ASSESSMENT="${GUIDE_BOARD_ACCEPT_ASSESSMENT:-../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json}"
|
||||||
|
OUTPUT_DIR="${GUIDE_BOARD_ACCEPT_OUTPUT_DIR:-${TMPDIR:-/tmp}/guide-board-external-extension-acceptance}"
|
||||||
|
PLAN_OUTPUT="${GUIDE_BOARD_ACCEPT_PLAN_OUTPUT:-$OUTPUT_DIR/plan.json}"
|
||||||
|
RUN_DIR="${GUIDE_BOARD_ACCEPT_RUN_DIR:-$OUTPUT_DIR/open-cmis-tck-baseline}"
|
||||||
|
PYTHON="${PYTHON:-python3}"
|
||||||
|
|
||||||
|
case "$MODE" in
|
||||||
|
plan|run)
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "Usage: $0 [plan|run]" >&2
|
||||||
|
exit 64
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
export PYTHONPATH="$ROOT_DIR/src${PYTHONPATH:+:$PYTHONPATH}"
|
||||||
|
|
||||||
|
if [ ! -f "$EXTENSION_DIR/extension.json" ]; then
|
||||||
|
echo "ERROR: extension manifest not found at $EXTENSION_DIR/extension.json" >&2
|
||||||
|
exit 66
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$OUTPUT_DIR"
|
||||||
|
|
||||||
|
echo "==> Listing discovered extensions"
|
||||||
|
"$PYTHON" -m guide_board \
|
||||||
|
--root "$ROOT_DIR" \
|
||||||
|
--extension-dir "$EXTENSION_DIR" \
|
||||||
|
extensions list
|
||||||
|
|
||||||
|
echo "==> Validating extension manifests"
|
||||||
|
"$PYTHON" -m guide_board \
|
||||||
|
--root "$ROOT_DIR" \
|
||||||
|
--extension-dir "$EXTENSION_DIR" \
|
||||||
|
extensions validate
|
||||||
|
|
||||||
|
echo "==> Validating target profile"
|
||||||
|
"$PYTHON" -m guide_board \
|
||||||
|
--root "$ROOT_DIR" \
|
||||||
|
--extension-dir "$EXTENSION_DIR" \
|
||||||
|
profile validate-target "$TARGET"
|
||||||
|
|
||||||
|
echo "==> Validating assessment profile"
|
||||||
|
"$PYTHON" -m guide_board \
|
||||||
|
--root "$ROOT_DIR" \
|
||||||
|
--extension-dir "$EXTENSION_DIR" \
|
||||||
|
profile validate-assessment "$ASSESSMENT"
|
||||||
|
|
||||||
|
echo "==> Generating run plan"
|
||||||
|
"$PYTHON" -m guide_board \
|
||||||
|
--root "$ROOT_DIR" \
|
||||||
|
--extension-dir "$EXTENSION_DIR" \
|
||||||
|
plan \
|
||||||
|
--target "$TARGET" \
|
||||||
|
--assessment "$ASSESSMENT" \
|
||||||
|
--output "$PLAN_OUTPUT"
|
||||||
|
|
||||||
|
if [ "$MODE" = "plan" ]; then
|
||||||
|
echo "External extension acceptance plan passed."
|
||||||
|
echo "Plan output: $PLAN_OUTPUT"
|
||||||
|
echo "Run live acceptance with: $0 run"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "==> Running live assessment"
|
||||||
|
"$PYTHON" -m guide_board \
|
||||||
|
--root "$ROOT_DIR" \
|
||||||
|
--extension-dir "$EXTENSION_DIR" \
|
||||||
|
run \
|
||||||
|
--target "$TARGET" \
|
||||||
|
--assessment "$ASSESSMENT" \
|
||||||
|
--output-dir "$RUN_DIR"
|
||||||
|
|
||||||
|
echo "==> Verifying retained run artifacts"
|
||||||
|
for path in \
|
||||||
|
"$RUN_DIR/run.json" \
|
||||||
|
"$RUN_DIR/plan.json" \
|
||||||
|
"$RUN_DIR/retention-summary.json" \
|
||||||
|
"$RUN_DIR/normalized/evidence.json" \
|
||||||
|
"$RUN_DIR/normalized/findings.json" \
|
||||||
|
"$RUN_DIR/normalized/mappings.json" \
|
||||||
|
"$RUN_DIR/reports/assessment-package.json" \
|
||||||
|
"$RUN_DIR/reports/report.md"
|
||||||
|
do
|
||||||
|
if [ ! -f "$path" ]; then
|
||||||
|
echo "ERROR: expected artifact missing: $path" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "==> Reviewing retained report paths"
|
||||||
|
"$PYTHON" -m guide_board \
|
||||||
|
--root "$ROOT_DIR" \
|
||||||
|
runs report \
|
||||||
|
--runs-dir "$OUTPUT_DIR"
|
||||||
|
|
||||||
|
echo "External extension live acceptance passed."
|
||||||
|
echo "Run artifacts: $RUN_DIR"
|
||||||
@@ -4,7 +4,7 @@ type: workplan
|
|||||||
title: "Assessment Operations Baseline"
|
title: "Assessment Operations Baseline"
|
||||||
repo: guide-board
|
repo: guide-board
|
||||||
domain: markitect
|
domain: markitect
|
||||||
status: active
|
status: completed
|
||||||
owner: codex
|
owner: codex
|
||||||
planning_priority: high
|
planning_priority: high
|
||||||
planning_order: 2
|
planning_order: 2
|
||||||
@@ -174,7 +174,7 @@ Progress:
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: GUIDE-BOARD-WP-0002-T006
|
id: GUIDE-BOARD-WP-0002-T006
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "65fbf1df-caef-40f6-abee-8308daf27fbc"
|
state_hub_task_id: "65fbf1df-caef-40f6-abee-8308daf27fbc"
|
||||||
```
|
```
|
||||||
@@ -187,6 +187,13 @@ Acceptance:
|
|||||||
- Cover extension validation, target/assessment profile validation, plan
|
- Cover extension validation, target/assessment profile validation, plan
|
||||||
generation, run execution, and result review.
|
generation, run execution, and result review.
|
||||||
|
|
||||||
|
Progress:
|
||||||
|
|
||||||
|
- Added `docs/EXTERNAL-EXTENSION-ACCEPTANCE.md`.
|
||||||
|
- Added `scripts/external_extension_acceptance.sh`.
|
||||||
|
- Verified `open-cmis-tck` extension validation, profile validation, and plan
|
||||||
|
generation in scripted plan mode.
|
||||||
|
|
||||||
## Definition Of Done
|
## Definition Of Done
|
||||||
|
|
||||||
- The repo has a clear assessment operations guide.
|
- The repo has a clear assessment operations guide.
|
||||||
|
|||||||
Reference in New Issue
Block a user