# Assessment Operations Status: draft Created: 2026-05-15 ## Purpose This document is the generic operator path for running guide-board assessments. It covers the local CLI and the dependency-light local service API. Extension repositories can provide domain-specific profiles and runtime setup, but the run, status, and result contracts are owned here. Guide-board prepares structured evidence and assessment packages. It does not issue certifications, provide audit assurance, or replace an accredited assessment process. ## Inputs Every run needs: - a target profile, - an assessment profile, - any extension repository roots required by the assessment profile, - optional credentials or mounted harness assets referenced by the target or assessment profile, - an output directory for run artifacts. The target profile describes the candidate system or artifact being assessed. The assessment profile selects frameworks, extensions, check groups, runtime policy, waivers, expectations, and output policy. ## CLI Flow Run from the guide-board repository: ```sh cd /home/worsch/guide-board PYTHONPATH=src python3 -m guide_board extensions validate PYTHONPATH=src python3 -m guide_board profile validate-target profiles/targets/sample-repository.json PYTHONPATH=src python3 -m guide_board profile validate-assessment profiles/assessments/sample-noop.json PYTHONPATH=src python3 -m guide_board plan \ --target profiles/targets/sample-repository.json \ --assessment profiles/assessments/sample-noop.json PYTHONPATH=src python3 -m guide_board run \ --target profiles/targets/sample-repository.json \ --assessment profiles/assessments/sample-noop.json \ --output-dir runs/sample-noop ``` For an external extension repository, pass `--extension-dir` before the subcommand: ```sh cd /home/worsch/guide-board PYTHONPATH=src python3 -m guide_board \ --extension-dir ../open-cmis-tck \ run \ --target ../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json \ --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \ --output-dir runs/open-cmis-tck-baseline ``` The same extension roots can be provided through `GUIDE_BOARD_EXTENSION_PATHS` when a wrapper script or container entrypoint should keep commands shorter. For the repeatable external extension acceptance path, including validation, planning, live execution, and retained result review, see `docs/EXTERNAL-EXTENSION-ACCEPTANCE.md`. For extension-author contracts such as profile schema descriptors and normalizer plug-ins, see `docs/EXTENSION-SDK.md`. ## CLI Results A completed CLI command prints a JSON result with: - `status`: assessment outcome, - `run_id`: generated run identifier, - `run_dir`: output directory, - `assessment_package`: JSON assessment package path, - `report`: Markdown report path, - `retention_summary`: compact durable summary path. The output directory uses this contract: ```text run.json plan.json retention-summary.json normalized/evidence.json normalized/findings.json normalized/mappings.json reports/assessment-package.json reports/report.md artifacts/ ``` Use the retained run helpers for history: ```sh PYTHONPATH=src python3 -m guide_board runs list --runs-dir runs PYTHONPATH=src python3 -m guide_board runs latest --runs-dir runs \ --target sample-repository \ --assessment sample-noop-assessment PYTHONPATH=src python3 -m guide_board runs report --runs-dir runs \ --target sample-repository \ --assessment sample-noop-assessment PYTHONPATH=src python3 -m guide_board runs trend --runs-dir runs PYTHONPATH=src python3 -m guide_board runs gate --runs-dir runs ``` ## Local Service Flow Start the service from the guide-board repository: ```sh cd /home/worsch/guide-board PYTHONPATH=src python3 -m guide_board \ --extension-dir ../open-cmis-tck \ serve --host 127.0.0.1 --port 8080 ``` Check health: ```sh curl -sf http://127.0.0.1:8080/health | python3 -m json.tool ``` Start a run: ```sh curl -sf -X POST http://127.0.0.1:8080/runs \ -H "Content-Type: application/json" \ -d '{ "target": "../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json", "assessment": "../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json", "output_dir": "runs/open-cmis-tck-baseline" }' | python3 -m json.tool ``` The response includes `job_id`. Inspect job status: ```sh curl -sf http://127.0.0.1:8080/runs/JOB_ID | python3 -m json.tool ``` Fetch reports after the job status is `succeeded`: ```sh curl -sf http://127.0.0.1:8080/runs/JOB_ID/reports | python3 -m json.tool ``` Service job state is currently in memory for the running service process. Run artifacts are durable in the output directory and can still be inspected after a service restart. See `docs/SERVICE-JOB-DURABILITY.md` for the restart and recovery contract. ## Status Vocabulary Service jobs use transport status: - `queued` - `running` - `succeeded` - `failed` Assessment runs use evidence status: - `completed` - `failed` - `blocked` - `infrastructure_error` A service job can be `succeeded` while the assessment status is `blocked` or `infrastructure_error`. That means guide-board completed the run machinery and produced evidence explaining why the assessment could not proceed cleanly. Individual evidence items use: - `pass` - `warning` - `fail` - `manual` - `skipped` - `blocked` - `not_applicable` - `expected_gap` - `infrastructure_error` ## Candidate System Checklist Before starting a run against candidate software, confirm: - the candidate service, repository, or artifact is in the state expected by the target profile, - endpoint URLs in the target profile are reachable from the guide-board process or container, - credentials are referenced through environment variables or mounted files, not committed into profiles, - the assessment profile selects only check groups that are safe for the candidate environment, - the output directory is outside source-controlled paths or is ignored. Candidate repositories should provide their own short handoff document with the startup command, profile path, required credentials, and expected endpoint URL. See `docs/CANDIDATE-HANDOFF.md` and `docs/templates/CANDIDATE-ASSESSMENT-HANDOFF.md` for the reusable contract.