coulomb/guide-board
2
0
Fork 0
generated from coulomb/repo-seed
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Close bootstrap and start assessment operations

Browse Source
This commit is contained in:
Bernd Worsch
2026-05-15 12:33:21 +02:00
parent 4357452af1
commit 2a7da8c04b
4 changed files with 360 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
README.md
View File

@@ -50,6 +50,7 @@ See:
- [INTENT.md](INTENT.md)
- [docs/ARCHITECTURE-BLUEPRINT.md](docs/ARCHITECTURE-BLUEPRINT.md)
- [docs/ASSESSMENT-OPERATIONS.md](docs/ASSESSMENT-OPERATIONS.md)
- [docs/COMPLIANCE-EVIDENCE-PACKS.md](docs/COMPLIANCE-EVIDENCE-PACKS.md)
- [docs/CONTAINER.md](docs/CONTAINER.md)
- [docs/EXTENSION-SDK.md](docs/EXTENSION-SDK.md)

193
docs/ASSESSMENT-OPERATIONS.md Normal file
View File

@@ -0,0 +1,193 @@
# Assessment Operations
Status: draft
Created: 2026-05-15
## Purpose
This document is the generic operator path for running guide-board assessments.
It covers the local CLI and the dependency-light local service API. Extension
repositories can provide domain-specific profiles and runtime setup, but the
run, status, and result contracts are owned here.
Guide-board prepares structured evidence and assessment packages. It does not
issue certifications, provide audit assurance, or replace an accredited
assessment process.
## Inputs
Every run needs:
- a target profile,
- an assessment profile,
- any extension repository roots required by the assessment profile,
- optional credentials or mounted harness assets referenced by the target or
assessment profile,
- an output directory for run artifacts.
The target profile describes the candidate system or artifact being assessed.
The assessment profile selects frameworks, extensions, check groups, runtime
policy, waivers, expectations, and output policy.
## CLI Flow
Run from the guide-board repository:
```sh
cd /home/worsch/guide-board
PYTHONPATH=src python3 -m guide_board extensions validate
PYTHONPATH=src python3 -m guide_board profile validate-target profiles/targets/sample-repository.json
PYTHONPATH=src python3 -m guide_board profile validate-assessment profiles/assessments/sample-noop.json
PYTHONPATH=src python3 -m guide_board plan \
--target profiles/targets/sample-repository.json \
--assessment profiles/assessments/sample-noop.json
PYTHONPATH=src python3 -m guide_board run \
--target profiles/targets/sample-repository.json \
--assessment profiles/assessments/sample-noop.json \
--output-dir runs/sample-noop
```
For an external extension repository, pass `--extension-dir` before the
subcommand:
```sh
cd /home/worsch/guide-board
PYTHONPATH=src python3 -m guide_board \
--extension-dir ../open-cmis-tck \
run \
--target ../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json \
--assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \
--output-dir runs/open-cmis-tck-baseline
```
The same extension roots can be provided through `GUIDE_BOARD_EXTENSION_PATHS`
when a wrapper script or container entrypoint should keep commands shorter.
## CLI Results
A completed CLI command prints a JSON result with:
- `status`: assessment outcome,
- `run_id`: generated run identifier,
- `run_dir`: output directory,
- `assessment_package`: JSON assessment package path,
- `report`: Markdown report path,
- `retention_summary`: compact durable summary path.
The output directory uses this contract:
```text
run.json
plan.json
retention-summary.json
normalized/evidence.json
normalized/findings.json
normalized/mappings.json
reports/assessment-package.json
reports/report.md
artifacts/
```
Use the retained run helpers for history:
```sh
PYTHONPATH=src python3 -m guide_board runs list --runs-dir runs
PYTHONPATH=src python3 -m guide_board runs trend --runs-dir runs
PYTHONPATH=src python3 -m guide_board runs gate --runs-dir runs
```
## Local Service Flow
Start the service from the guide-board repository:
```sh
cd /home/worsch/guide-board
PYTHONPATH=src python3 -m guide_board \
--extension-dir ../open-cmis-tck \
serve --host 127.0.0.1 --port 8080
```
Check health:
```sh
curl -sf http://127.0.0.1:8080/health | python3 -m json.tool
```
Start a run:
```sh
curl -sf -X POST http://127.0.0.1:8080/runs \
-H "Content-Type: application/json" \
-d '{
"target": "../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json",
"assessment": "../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json",
"output_dir": "runs/open-cmis-tck-baseline"
}' | python3 -m json.tool
```
The response includes `job_id`.
Inspect job status:
```sh
curl -sf http://127.0.0.1:8080/runs/JOB_ID | python3 -m json.tool
```
Fetch reports after the job status is `succeeded`:
```sh
curl -sf http://127.0.0.1:8080/runs/JOB_ID/reports | python3 -m json.tool
```
Service job state is currently in memory for the running service process. Run
artifacts are durable in the output directory and can still be inspected after a
service restart.
## Status Vocabulary
Service jobs use transport status:
- `queued`
- `running`
- `succeeded`
- `failed`
Assessment runs use evidence status:
- `completed`
- `failed`
- `blocked`
- `infrastructure_error`
A service job can be `succeeded` while the assessment status is `blocked` or
`infrastructure_error`. That means guide-board completed the run machinery and
produced evidence explaining why the assessment could not proceed cleanly.
Individual evidence items use:
- `pass`
- `warning`
- `fail`
- `manual`
- `skipped`
- `blocked`
- `not_applicable`
- `expected_gap`
- `infrastructure_error`
## Candidate System Checklist
Before starting a run against candidate software, confirm:
- the candidate service, repository, or artifact is in the state expected by
the target profile,
- endpoint URLs in the target profile are reachable from the guide-board
process or container,
- credentials are referenced through environment variables or mounted files,
not committed into profiles,
- the assessment profile selects only check groups that are safe for the
candidate environment,
- the output directory is outside source-controlled paths or is ignored.
Candidate repositories should provide their own short handoff document with the
startup command, profile path, required credentials, and expected endpoint URL.

4
workplans/GUIDE-BOARD-WP-0001-bootstrapping.md
View File

@@ -4,12 +4,12 @@ type: workplan
title: "Guide Board Bootstrapping"
repo: guide-board
domain: markitect
status: active
status: completed
owner: codex
planning_priority: high
planning_order: 1
created: "2026-05-07"
updated: "2026-05-07"
updated: "2026-05-15"
supersedes:
- "OPEN-CMIS-TCK-WP-0001"
state_hub_workstream_id: "1d812b7d-74f0-4d71-86a7-0f390b22daf7"

164
workplans/GUIDE-BOARD-WP-0002-assessment-operations-baseline.md Normal file
View File

@@ -0,0 +1,164 @@
---
id: GUIDE-BOARD-WP-0002
type: workplan
title: "Assessment Operations Baseline"
repo: guide-board
domain: markitect
status: active
owner: codex
planning_priority: high
planning_order: 2
created: "2026-05-15"
updated: "2026-05-15"
state_hub_workstream_id: "fc5b1573-91b2-4a19-b6a9-dd4d17057d9b"
---
# GUIDE-BOARD-WP-0002: Assessment Operations Baseline
## Purpose
Turn the bootstrapped framework into an operator-ready assessment tool. The
next layer should make it obvious how a candidate software repository or
operator starts an assessment, follows status, retrieves results, and packages
evidence without needing to understand guide-board internals.
## Background
`GUIDE-BOARD-WP-0001` established the core contracts: schemas, extension
discovery, profile validation, planning, execution, service transport,
container baseline, retention summaries, and the first external extension path
with `open-cmis-tck`.
The next gap is operational clarity. A user should be able to start from a
candidate system and know:
- what target and assessment profiles are required,
- which command or HTTP endpoint starts the run,
- how status differs between service jobs and assessment outcomes,
- where reports, normalized evidence, raw artifacts, and retention summaries
are written,
- what a candidate repository should document when it wants to be assessed.
## Boundary
This workplan owns generic guide-board assessment operation. It may reference
`open-cmis-tck` as a concrete example, but CMIS-specific harness behavior,
target profiles, and runtime dependencies remain owned by the external
`open-cmis-tck` extension repository.
## D2.1 - Assessment Operations Runbook
```task
id: GUIDE-BOARD-WP-0002-T001
status: done
priority: high
state_hub_task_id: "4592bb4f-096a-49e4-8785-06f9c3f619ac"
```
Acceptance:
- Add a generic operator document for CLI and local service assessment flows.
- Document start, status, report retrieval, output directories, and result
states.
- Include the external-extension shape without making CMIS the root product.
- Link the runbook from README.
Progress:
- Added `docs/ASSESSMENT-OPERATIONS.md`.
- Linked the runbook from README.
## D2.2 - Candidate Repository Handoff Contract
```task
id: GUIDE-BOARD-WP-0002-T002
status: todo
priority: high
state_hub_task_id: "7d1df7d0-49cd-487f-a507-9e558a90af31"
```
Acceptance:
- Define the minimal instruction file a candidate repository should provide
when it wants guide-board assessment support.
- Cover required endpoint/profile facts, credential references, startup
command, expected runtime state, and where candidate-specific results should
be referenced.
- Provide a template that candidate repos can copy without importing
guide-board internals.
## D2.3 - Run History And Result UX
```task
id: GUIDE-BOARD-WP-0002-T003
status: todo
priority: medium
state_hub_task_id: "ce18a2dc-7cc9-4fff-9272-5333b6118030"
```
Acceptance:
- Make it easy to find the latest run for a target and assessment pair.
- Document or add commands for listing run reports and opening the latest
report paths.
- Preserve the existing run directory contract and retention summary model.
## D2.4 - Service Job Durability Contract
```task
id: GUIDE-BOARD-WP-0002-T004
status: todo
priority: medium
state_hub_task_id: "10e4003c-dc11-4a8e-aecc-7815559ac439"
```
Acceptance:
- Decide whether local service job state remains intentionally in-memory or
gains a small durable index.
- If in-memory remains the baseline, document restart semantics and the durable
fallback through run directories.
- If durable indexing is added, keep it dependency-light and reconstructable
from retained run artifacts.
## D2.5 - Container Smoke Acceptance
```task
id: GUIDE-BOARD-WP-0002-T005
status: todo
priority: medium
state_hub_task_id: "9e2e7fa7-8a97-4f07-9925-e637e0366003"
```
Acceptance:
- Add or document a reproducible container smoke check for the bundled sample
assessment.
- Verify mounted output directories receive run artifacts.
- Keep restricted or extension-specific harness assets outside the core image.
## D2.6 - External Extension Acceptance Path
```task
id: GUIDE-BOARD-WP-0002-T006
status: todo
priority: high
state_hub_task_id: "65fbf1df-caef-40f6-abee-8308daf27fbc"
```
Acceptance:
- Define a repeatable acceptance path for an external extension repository.
- Use `open-cmis-tck` as the first example without moving CMIS-specific logic
back into the core.
- Cover extension validation, target/assessment profile validation, plan
generation, run execution, and result review.
## Definition Of Done
- The repo has a clear assessment operations guide.
- Candidate repositories know what handoff information to provide.
- Operators can start runs, inspect status, and retrieve results through CLI or
local service.
- The service and container baselines have documented operational limits.
- The next external extension acceptance workflow is repeatable.