generated from coulomb/repo-seed
205 lines
6.1 KiB
Markdown
205 lines
6.1 KiB
Markdown
---
|
|
id: GUIDE-BOARD-WP-0002
|
|
type: workplan
|
|
title: "Assessment Operations Baseline"
|
|
repo: guide-board
|
|
domain: markitect
|
|
status: completed
|
|
owner: codex
|
|
planning_priority: high
|
|
planning_order: 2
|
|
created: "2026-05-15"
|
|
updated: "2026-05-15"
|
|
state_hub_workstream_id: "fc5b1573-91b2-4a19-b6a9-dd4d17057d9b"
|
|
---
|
|
|
|
# GUIDE-BOARD-WP-0002: Assessment Operations Baseline
|
|
|
|
## Purpose
|
|
|
|
Turn the bootstrapped framework into an operator-ready assessment tool. The
|
|
next layer should make it obvious how a candidate software repository or
|
|
operator starts an assessment, follows status, retrieves results, and packages
|
|
evidence without needing to understand guide-board internals.
|
|
|
|
## Background
|
|
|
|
`GUIDE-BOARD-WP-0001` established the core contracts: schemas, extension
|
|
discovery, profile validation, planning, execution, service transport,
|
|
container baseline, retention summaries, and the first external extension path
|
|
with `open-cmis-tck`.
|
|
|
|
The next gap is operational clarity. A user should be able to start from a
|
|
candidate system and know:
|
|
|
|
- what target and assessment profiles are required,
|
|
- which command or HTTP endpoint starts the run,
|
|
- how status differs between service jobs and assessment outcomes,
|
|
- where reports, normalized evidence, raw artifacts, and retention summaries
|
|
are written,
|
|
- what a candidate repository should document when it wants to be assessed.
|
|
|
|
## Boundary
|
|
|
|
This workplan owns generic guide-board assessment operation. It may reference
|
|
`open-cmis-tck` as a concrete example, but CMIS-specific harness behavior,
|
|
target profiles, and runtime dependencies remain owned by the external
|
|
`open-cmis-tck` extension repository.
|
|
|
|
## D2.1 - Assessment Operations Runbook
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0002-T001
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "4592bb4f-096a-49e4-8785-06f9c3f619ac"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Add a generic operator document for CLI and local service assessment flows.
|
|
- Document start, status, report retrieval, output directories, and result
|
|
states.
|
|
- Include the external-extension shape without making CMIS the root product.
|
|
- Link the runbook from README.
|
|
|
|
Progress:
|
|
|
|
- Added `docs/ASSESSMENT-OPERATIONS.md`.
|
|
- Linked the runbook from README.
|
|
|
|
## D2.2 - Candidate Repository Handoff Contract
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0002-T002
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "7d1df7d0-49cd-487f-a507-9e558a90af31"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Define the minimal instruction file a candidate repository should provide
|
|
when it wants guide-board assessment support.
|
|
- Cover required endpoint/profile facts, credential references, startup
|
|
command, expected runtime state, and where candidate-specific results should
|
|
be referenced.
|
|
- Provide a template that candidate repos can copy without importing
|
|
guide-board internals.
|
|
|
|
Progress:
|
|
|
|
- Added `docs/CANDIDATE-HANDOFF.md`.
|
|
- Added `docs/templates/CANDIDATE-ASSESSMENT-HANDOFF.md`.
|
|
- Linked the handoff contract from README and the assessment operations guide.
|
|
|
|
## D2.3 - Run History And Result UX
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0002-T003
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "ce18a2dc-7cc9-4fff-9272-5333b6118030"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Make it easy to find the latest run for a target and assessment pair.
|
|
- Document or add commands for listing run reports and opening the latest
|
|
report paths.
|
|
- Preserve the existing run directory contract and retention summary model.
|
|
|
|
Progress:
|
|
|
|
- Added `guide_board.retention.select_retained_run`.
|
|
- Added `guide_board.retention.retained_run_report_paths`.
|
|
- Added `guide-board runs latest` and `guide-board runs report`.
|
|
- Documented the new commands in `docs/ASSESSMENT-OPERATIONS.md`.
|
|
|
|
## D2.4 - Service Job Durability Contract
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0002-T004
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "10e4003c-dc11-4a8e-aecc-7815559ac439"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Decide whether local service job state remains intentionally in-memory or
|
|
gains a small durable index.
|
|
- If in-memory remains the baseline, document restart semantics and the durable
|
|
fallback through run directories.
|
|
- If durable indexing is added, keep it dependency-light and reconstructable
|
|
from retained run artifacts.
|
|
|
|
Decision:
|
|
|
|
- Keep local service job state intentionally in-memory for the baseline.
|
|
- Treat run directories and `retention-summary.json` as the durable recovery
|
|
source.
|
|
|
|
Progress:
|
|
|
|
- Added `docs/SERVICE-JOB-DURABILITY.md`.
|
|
- Linked the contract from README, the local service API docs, and the
|
|
assessment operations guide.
|
|
|
|
## D2.5 - Container Smoke Acceptance
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0002-T005
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "9e2e7fa7-8a97-4f07-9925-e637e0366003"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Add or document a reproducible container smoke check for the bundled sample
|
|
assessment.
|
|
- Verify mounted output directories receive run artifacts.
|
|
- Keep restricted or extension-specific harness assets outside the core image.
|
|
|
|
Progress:
|
|
|
|
- Added `scripts/container_smoke.sh`.
|
|
- Documented the smoke check in `docs/CONTAINER.md`.
|
|
- Verified the smoke check with Docker; mounted outputs received run metadata,
|
|
normalized evidence, mappings, findings, assessment package, and Markdown
|
|
report.
|
|
|
|
## D2.6 - External Extension Acceptance Path
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0002-T006
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "65fbf1df-caef-40f6-abee-8308daf27fbc"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Define a repeatable acceptance path for an external extension repository.
|
|
- Use `open-cmis-tck` as the first example without moving CMIS-specific logic
|
|
back into the core.
|
|
- Cover extension validation, target/assessment profile validation, plan
|
|
generation, run execution, and result review.
|
|
|
|
Progress:
|
|
|
|
- Added `docs/EXTERNAL-EXTENSION-ACCEPTANCE.md`.
|
|
- Added `scripts/external_extension_acceptance.sh`.
|
|
- Verified `open-cmis-tck` extension validation, profile validation, and plan
|
|
generation in scripted plan mode.
|
|
|
|
## Definition Of Done
|
|
|
|
- The repo has a clear assessment operations guide.
|
|
- Candidate repositories know what handoff information to provide.
|
|
- Operators can start runs, inspect status, and retrieve results through CLI or
|
|
local service.
|
|
- The service and container baselines have documented operational limits.
|
|
- The next external extension acceptance workflow is repeatable.
|