generated from coulomb/repo-seed
156 lines
5.3 KiB
Markdown
156 lines
5.3 KiB
Markdown
---
|
|
id: GUIDE-BOARD-WP-0005
|
|
type: workplan
|
|
title: "Challenge And Exclusion Handling"
|
|
repo: guide-board
|
|
domain: markitect
|
|
status: completed
|
|
owner: codex
|
|
planning_priority: high
|
|
planning_order: 5
|
|
created: "2026-05-15"
|
|
updated: "2026-05-16"
|
|
state_hub_workstream_id: "fb11e1c7-6c0c-4ec7-a163-da98b2fe9f8f"
|
|
---
|
|
|
|
# GUIDE-BOARD-WP-0005: Challenge And Exclusion Handling
|
|
|
|
## Purpose
|
|
|
|
Represent authority exclusions, extension challenges, target expectations,
|
|
waivers, and defects as distinct review concepts. Guide-board already supports
|
|
expectations and waivers, but real assessments also need a way to record that a
|
|
test was excluded by an authority, challenged as invalid or mis-mapped, or
|
|
identified as a target defect.
|
|
|
|
## Background
|
|
|
|
Conformance work often includes dispute and exclusion paths. A failed check may
|
|
be a product issue, a harness issue, an unsupported optional feature, a known
|
|
waived risk, an authority-approved exclusion, or a local challenge awaiting
|
|
review. Reports need to make those distinctions visible instead of flattening
|
|
everything into pass, fail, blocked, or waived.
|
|
|
|
## Boundary
|
|
|
|
This workplan owns guide-board's generic challenge and exclusion model. It does
|
|
not decide authority-specific challenge rules, certification program policy, or
|
|
whether a challenge is valid. Extensions may provide authority-specific fields,
|
|
but the core should preserve them without embedding domain policy.
|
|
|
|
## D5.1 - Challenge And Exclusion Data Contracts
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0005-T001
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "6ff4e6f7-bce6-4e7f-a5af-e0c67cfa7e55"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Define schemas for authority exclusions and extension challenges.
|
|
- Distinguish challenge/exclusion records from expectations and waivers.
|
|
- Support links to requirement refs, check IDs, evidence IDs, authority source
|
|
refs, owners, review status, rationale, expiry or review dates, and native
|
|
challenge IDs where available.
|
|
- Keep the data contract usable by executable harnesses, hosted suites, and
|
|
procedural packs.
|
|
|
|
Progress:
|
|
|
|
- Added `docs/schemas/challenge-set.schema.json` and
|
|
`docs/schemas/exclusion-set.schema.json`.
|
|
- Added optional `challenges_ref` and `exclusions_ref` assessment profile
|
|
fields.
|
|
- Supported requirement, check, evidence, result, classification, authority
|
|
source, owner, review status, rationale, review date, expiry, native ID, and
|
|
metadata fields.
|
|
|
|
## D5.2 - Policy Application And Finding Annotation
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0005-T002
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "fd384bd3-40c4-4344-8b7d-cb123dbf2cac"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Load challenge and exclusion references from assessment profiles.
|
|
- Annotate findings and evidence with challenge or exclusion state without
|
|
silently hiding unexpected failures.
|
|
- Preserve separate counts for expected findings, waived findings, challenged
|
|
findings, authority exclusions, and unresolved defects.
|
|
- Add tests that prove challenge and exclusion records affect reporting without
|
|
corrupting gate semantics.
|
|
|
|
Progress:
|
|
|
|
- Loaded challenge and exclusion refs through the policy layer.
|
|
- Annotated findings with challenge refs, exclusion refs, and review status.
|
|
- Annotated matching evidence with review refs.
|
|
- Kept default `unexpected_findings` gate semantics visible unless a finding is
|
|
separately expected or waived.
|
|
- Added tests proving challenged and excluded findings remain gate-visible.
|
|
|
|
## D5.3 - Report Visibility And Review Workflow
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0005-T003
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "791071c0-8a9a-462b-83b3-75548bb8524f"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Show challenge, exclusion, waiver, expectation, and defect distinctions in
|
|
Markdown and JSON assessment packages.
|
|
- Make unresolved review items easy to find in retained run summaries.
|
|
- Provide CLI history or report helpers that expose review state for the latest
|
|
run.
|
|
- Document how an operator should treat challenged or excluded findings.
|
|
|
|
Progress:
|
|
|
|
- Added Markdown report review summaries.
|
|
- Added challenge, exclusion, unresolved defect, and unresolved review counts to
|
|
retention summaries and trend projections.
|
|
- Included applied challenge and exclusion records in JSON assessment packages.
|
|
- Exposed review counts through existing retained run helpers.
|
|
|
|
## D5.4 - Tests And Documentation
|
|
|
|
```task
|
|
id: GUIDE-BOARD-WP-0005-T004
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "43b966da-af8d-479b-93bd-6b6741fdab37"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Add schema tests and policy tests for challenges and exclusions.
|
|
- Add a sample or SDK fixture scenario that produces at least one challenged or
|
|
excluded finding.
|
|
- Update assessment operations, extension SDK, and compliance evidence pack docs.
|
|
- Keep certification boundary language explicit.
|
|
|
|
Progress:
|
|
|
|
- Added focused schema and policy tests through a fixture extension scenario.
|
|
- Updated assessment operations, extension SDK, compliance evidence pack, and
|
|
architecture docs.
|
|
- Kept boundary language explicit: challenges and exclusions are review state,
|
|
not certification conclusions.
|
|
|
|
## Definition Of Done
|
|
|
|
- The core has separate, tested concepts for expectations, waivers, challenges,
|
|
authority exclusions, and defects.
|
|
- Reports surface those concepts without overstating certification conclusions.
|
|
- Retained summaries expose unresolved review work.
|
|
- Extension authors know how to supply authority-specific challenge metadata.
|