generated from coulomb/repo-seed
core architecture blueprint
This commit is contained in:
4
.gitignore
vendored
4
.gitignore
vendored
@@ -176,3 +176,7 @@ cython_debug/
|
||||
|
||||
# Editor swap files
|
||||
*.swp
|
||||
|
||||
# Guide Board local generated outputs
|
||||
/runs/
|
||||
/reports/
|
||||
|
||||
@@ -148,7 +148,8 @@ capability groups, and produce guide-board-compatible evidence reports.
|
||||
Initial candidate extensions are registered in `extensions/CANDIDATES.md`. They
|
||||
include official or authority-backed conformance harness patterns from OGC, the
|
||||
OpenID Foundation, CNCF Kubernetes, W3C/WHATWG web-platform-tests, Khronos CTS,
|
||||
NIST ACVP, ONC/HL7 FHIR Inferno, Jakarta EE TCK, OPC UA CTT, and CMIS/OpenCMIS.
|
||||
NIST ACVP, ONC/HL7 FHIR Inferno, Jakarta EE TCK, OPC UA CTT, NIST
|
||||
SCAP/OpenSCAP, NIST OSCAL, CIS-CAT Pro, OpenSSF Scorecard, and CMIS/OpenCMIS.
|
||||
|
||||
The point of studying these candidates is not to implement everything at once. It
|
||||
is to make the core architecture fit the real shapes of conformance work:
|
||||
|
||||
@@ -10,6 +10,7 @@ extensions.
|
||||
See:
|
||||
|
||||
- [INTENT.md](INTENT.md)
|
||||
- [docs/ARCHITECTURE-BLUEPRINT.md](docs/ARCHITECTURE-BLUEPRINT.md)
|
||||
- [extensions/CANDIDATES.md](extensions/CANDIDATES.md)
|
||||
- [extensions/open-cmis-tck/INTENT.md](extensions/open-cmis-tck/INTENT.md)
|
||||
- [workplans/GUIDE-BOARD-WP-0001-bootstrapping.md](workplans/GUIDE-BOARD-WP-0001-bootstrapping.md)
|
||||
|
||||
773
docs/ARCHITECTURE-BLUEPRINT.md
Normal file
773
docs/ARCHITECTURE-BLUEPRINT.md
Normal file
@@ -0,0 +1,773 @@
|
||||
# Guide Board Core Architecture Blueprint
|
||||
|
||||
Status: draft
|
||||
Created: 2026-05-07
|
||||
|
||||
## Purpose
|
||||
|
||||
This blueprint defines the first core architecture for `guide-board`: a
|
||||
certification and compliance preparation framework that can orchestrate
|
||||
extension-specific conformance harnesses, validators, repository-quality checks,
|
||||
and procedural evidence packs without embedding domain policy in the core.
|
||||
|
||||
The design is based on recurring patterns from official or authority-backed
|
||||
programs such as OGC TEAM Engine, OpenID Foundation Conformance Suite, CNCF
|
||||
Kubernetes Conformance, web-platform-tests, Khronos CTS, NIST ACVP, HL7/FHIR
|
||||
Inferno, Jakarta EE TCK, OPC UA CTT, NIST SCAP/OSCAL, CIS-CAT, and OpenSSF
|
||||
Scorecard.
|
||||
|
||||
## Research Lessons
|
||||
|
||||
### Suite Engine
|
||||
|
||||
Examples: OGC TEAM Engine, OpenCMIS TCK, Jakarta EE TCK.
|
||||
|
||||
Pattern:
|
||||
|
||||
- installable suites with their own test definitions,
|
||||
- command-line execution and sometimes web/API execution,
|
||||
- target-specific input forms or profiles,
|
||||
- raw logs plus structured result formats,
|
||||
- conformance classes or capability areas,
|
||||
- certification boundary outside normal self-testing.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
`guide-board` needs a runner bridge that can call external harnesses, capture
|
||||
artifacts, and normalize tool-specific result formats without making the harness
|
||||
part of the core.
|
||||
|
||||
Sources:
|
||||
|
||||
- [TEAM Engine](https://opengeospatial.github.io/teamengine/)
|
||||
- [TEAM Engine User Guide](https://opengeospatial.github.io/teamengine/users.html)
|
||||
- [Jakarta EE TCK Process](https://jakarta.ee/committees/specification/tckprocess/)
|
||||
- [OpenCMIS TCK package](https://chemistry.apache.org/java/javadoc/org/apache/chemistry/opencmis/tck/package-summary.html)
|
||||
|
||||
### Hosted Or Local Certification Suite
|
||||
|
||||
Examples: OpenID Foundation Conformance Suite, Inferno.
|
||||
|
||||
Pattern:
|
||||
|
||||
- open source suite,
|
||||
- hosted public/staging environments,
|
||||
- local Docker execution,
|
||||
- named test plans or test kits,
|
||||
- logs and public result pages,
|
||||
- fee or accredited-review boundary for formal certification.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
`guide-board` should model execution environment tiers, test plans, and
|
||||
certification submission packages separately from normal development runs.
|
||||
|
||||
Sources:
|
||||
|
||||
- [OpenID Conformance Suite](https://openid.net/certification/about-conformance-suite/)
|
||||
- [OpenID Certification](https://openid.net/certification/)
|
||||
- [Inferno Framework](https://inferno-framework.github.io/about/)
|
||||
- [Inferno Documentation](https://inferno-framework.github.io/docs)
|
||||
|
||||
### Submit-Results Program
|
||||
|
||||
Example: CNCF Kubernetes Conformance.
|
||||
|
||||
Pattern:
|
||||
|
||||
- vendors run the same open source conformance application used by the program,
|
||||
- result artifacts are submitted for review,
|
||||
- accepted results feed a public certification list,
|
||||
- users can rerun the same conformance application to confirm behavior.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
An assessment package should be a first-class artifact with source metadata,
|
||||
runner version, target identity, raw evidence, normalized results, and a review
|
||||
boundary suitable for downstream submission.
|
||||
|
||||
Source:
|
||||
|
||||
- [CNCF Certified Kubernetes Software Conformance](https://www.cncf.io/certification/software-conformance/)
|
||||
|
||||
### Protocol Validation Service
|
||||
|
||||
Example: NIST ACVP.
|
||||
|
||||
Pattern:
|
||||
|
||||
- authority-operated demo and production services,
|
||||
- client authentication,
|
||||
- machine-to-machine protocol,
|
||||
- generated test vectors and submitted responses,
|
||||
- validation tied to an external authority process.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
Some extensions will not run a local test suite. They will coordinate a session
|
||||
with an authority service. The core must support credential references, remote
|
||||
session IDs, generated inputs, submitted responses, and external verdicts.
|
||||
|
||||
Source:
|
||||
|
||||
- [NIST ACVP](https://pages.nist.gov/ACVP/)
|
||||
|
||||
### Web-Scale Shared Test Repository
|
||||
|
||||
Example: web-platform-tests.
|
||||
|
||||
Pattern:
|
||||
|
||||
- shared specification-linked test repository,
|
||||
- canonical manifest generation,
|
||||
- multiple test types including automated, reference, and manual tests,
|
||||
- local and public execution surfaces.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
`guide-board` check discovery should be manifest-driven where possible. It must
|
||||
support heterogeneous check types instead of assuming every check is a simple
|
||||
pass/fail command.
|
||||
|
||||
Sources:
|
||||
|
||||
- [web-platform-tests](https://web-platform-tests.org/)
|
||||
- [Writing Your Own Runner](https://web-platform-tests.org/running-tests/custom-runner.html)
|
||||
- [Running Tests from the Web](https://web-platform-tests.org/running-tests/from-web.html)
|
||||
|
||||
### Conformance Submission Package
|
||||
|
||||
Examples: Khronos Vulkan CTS and OpenXR CTS.
|
||||
|
||||
Pattern:
|
||||
|
||||
- automated and sometimes interactive test runs,
|
||||
- XML result files,
|
||||
- console output,
|
||||
- build and CTS version metadata,
|
||||
- explicit conformance statement,
|
||||
- trademark or adopter-program boundary.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
The guide-board assessment package should preserve both normalized evidence and
|
||||
the original submission-grade artifacts expected by an authority.
|
||||
|
||||
Sources:
|
||||
|
||||
- [Vulkan CTS Guide](https://docs.vulkan.org/guide/latest/vulkan_cts.html)
|
||||
- [OpenXR CTS Usage Guide](https://registry.khronos.org/OpenXR/conformance/cts_usage.html)
|
||||
|
||||
### Restricted Tool
|
||||
|
||||
Examples: OPC UA CTT, CIS-CAT Pro.
|
||||
|
||||
Pattern:
|
||||
|
||||
- official tool may be restricted to members, licensees, or controlled access,
|
||||
- tests are organized by profiles, facets, conformance units, benchmarks, or
|
||||
controls,
|
||||
- command-line execution may exist for automation,
|
||||
- redistribution is not allowed or not appropriate.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
`guide-board` must represent restricted harnesses as externally supplied runtime
|
||||
assets. The registry can describe how to integrate them, but the core and
|
||||
extensions must not vendor restricted tools or proprietary standard text.
|
||||
|
||||
Sources:
|
||||
|
||||
- [OPC UA Compliance Test Tool](https://opcfoundation.org/developer-tools/certification-test-tools/opc-ua-compliance-test-tool-uactt/)
|
||||
- [CIS-CAT Pro Assessor](https://www.cisecurity.org/cybersecurity-tools/cis-cat-pro)
|
||||
|
||||
### Security Configuration And Assessment Content
|
||||
|
||||
Examples: NIST SCAP, OpenSCAP, CIS-CAT Pro.
|
||||
|
||||
Pattern:
|
||||
|
||||
- machine-readable security configuration content,
|
||||
- profiles or tailored benchmarks,
|
||||
- local or remote system assessment,
|
||||
- automated and manual checks,
|
||||
- reports mapped to controls.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
`guide-board` must support content-driven validators where the extension supplies
|
||||
policy content and a scanner, not a fixed test suite. The evidence model must
|
||||
handle manual, automated, and partially automated checks.
|
||||
|
||||
Sources:
|
||||
|
||||
- [NIST SCAP](https://csrc.nist.gov/Projects/Security-Content-Automation-Protocol)
|
||||
- [NIST SCAP 1.3](https://csrc.nist.gov/projects/security-content-automation-protocol/scap-releases/scap-1-3)
|
||||
- [OpenSCAP](https://www.open-scap.org/)
|
||||
|
||||
### Assessment Data Interchange
|
||||
|
||||
Example: NIST OSCAL.
|
||||
|
||||
Pattern:
|
||||
|
||||
- layered machine-readable models for controls, implementation, assessment
|
||||
plans, assessment results, and remediation milestones,
|
||||
- multiple serializations such as JSON, XML, and YAML,
|
||||
- assessment results expressed relative to a system and controls.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
`guide-board` should keep its internal evidence model small, but design it so
|
||||
later OSCAL export is natural for compliance packs that need formal assessment
|
||||
interchange.
|
||||
|
||||
Source:
|
||||
|
||||
- [NIST OSCAL Layers and Models](https://pages.nist.gov/OSCAL/learn/concepts/layer/)
|
||||
|
||||
### Repository Quality And Supply Chain Scoring
|
||||
|
||||
Example: OpenSSF Scorecard.
|
||||
|
||||
Pattern:
|
||||
|
||||
- automated checks over source repositories,
|
||||
- score and risk level per check,
|
||||
- aggregate posture score,
|
||||
- remediation prompts,
|
||||
- CI and API integration.
|
||||
|
||||
Architecture lesson:
|
||||
|
||||
Repository quality packs should be normal extensions. A score is not a
|
||||
certification verdict; it is a normalized finding and trend signal.
|
||||
|
||||
Sources:
|
||||
|
||||
- [OpenSSF Scorecard](https://openssf.org/projects/scorecard/)
|
||||
- [Scorecard documentation](https://github.com/ossf/scorecard)
|
||||
|
||||
## Architecture Principles
|
||||
|
||||
- The core is extension-neutral.
|
||||
- Authority, framework, and harness versions are evidence, not prose.
|
||||
- Local CLI behavior is the execution source of truth.
|
||||
- Optional service APIs wrap the same contracts used by the CLI.
|
||||
- Restricted harnesses and proprietary standards are mounted or referenced, not
|
||||
redistributed.
|
||||
- Raw artifacts are preserved, but normalized evidence is the primary interface.
|
||||
- Every assessment package must state its certification boundary.
|
||||
- Manual, semi-automated, and fully automated checks all use the same evidence
|
||||
model.
|
||||
- Expected gaps and waivers never suppress unexpected failures silently.
|
||||
- Extension extraction to separate repositories should be possible without
|
||||
changing core contracts.
|
||||
|
||||
## Core Components
|
||||
|
||||
### Authority Catalog
|
||||
|
||||
Tracks source authorities, framework names, versions, official URLs, licensing
|
||||
posture, access constraints, certification boundaries, and lifecycle status.
|
||||
|
||||
### Extension Registry
|
||||
|
||||
Discovers installed or incubating extensions. Each extension declares:
|
||||
|
||||
- extension ID,
|
||||
- type,
|
||||
- supported frameworks,
|
||||
- source authority references,
|
||||
- profile schemas,
|
||||
- check groups,
|
||||
- runner or validator entry points,
|
||||
- normalizers,
|
||||
- mappings,
|
||||
- report fragments,
|
||||
- dependency and license posture.
|
||||
|
||||
### Profile Registry
|
||||
|
||||
Loads and validates target profiles and assessment profiles.
|
||||
|
||||
Target profiles describe the subject being assessed: repository, service,
|
||||
cluster, product, API, data archive, host, organization, process, or policy set.
|
||||
|
||||
Assessment profiles select frameworks, controls, check groups, expectations,
|
||||
waivers, output policies, and retention policies.
|
||||
|
||||
### Assessment Planner
|
||||
|
||||
Resolves an assessment profile into an executable run plan:
|
||||
|
||||
- selected extensions,
|
||||
- selected check groups,
|
||||
- required credentials,
|
||||
- preflight checks,
|
||||
- dependency checks,
|
||||
- execution order,
|
||||
- isolation and timeout policy,
|
||||
- artifact retention policy.
|
||||
|
||||
### Runner Bridge
|
||||
|
||||
Executes or coordinates extension checks.
|
||||
|
||||
Supported runner kinds:
|
||||
|
||||
- local command,
|
||||
- container command,
|
||||
- in-process validator,
|
||||
- remote protocol session,
|
||||
- hosted test-suite interaction,
|
||||
- manual evidence request,
|
||||
- imported result package.
|
||||
|
||||
### Artifact Store
|
||||
|
||||
Stores run artifacts by reference and checksum:
|
||||
|
||||
- raw logs,
|
||||
- XML/JSON/HTML reports,
|
||||
- screenshots or rendered documents,
|
||||
- authority submission files,
|
||||
- request/response transcripts,
|
||||
- input forms,
|
||||
- profile snapshots,
|
||||
- source lockfiles.
|
||||
|
||||
### Normalizer
|
||||
|
||||
Converts extension output into guide-board evidence records.
|
||||
|
||||
The normalizer should preserve native identifiers such as test case IDs,
|
||||
conformance class IDs, control IDs, profile IDs, benchmark IDs, or requirement
|
||||
references.
|
||||
|
||||
### Mapping Engine
|
||||
|
||||
Maps evidence to:
|
||||
|
||||
- capabilities,
|
||||
- controls,
|
||||
- conformance classes,
|
||||
- requirements,
|
||||
- policy questions,
|
||||
- repository quality dimensions,
|
||||
- scorecard dimensions.
|
||||
|
||||
Mappings belong to extensions or assessment packs, not the core.
|
||||
|
||||
### Expectation And Waiver Engine
|
||||
|
||||
Applies declared target posture after evidence normalization.
|
||||
|
||||
Use expectations for known optional behavior, unsupported-by-design features, and
|
||||
accepted gaps.
|
||||
|
||||
Use waivers for time-bounded exceptions with owner, reason, expiry, and review
|
||||
metadata.
|
||||
|
||||
### Report Builder
|
||||
|
||||
Builds human and machine-readable outputs:
|
||||
|
||||
- compact JSON assessment package,
|
||||
- Markdown summary,
|
||||
- extension-specific fragments,
|
||||
- submission package manifest,
|
||||
- trend summaries,
|
||||
- future OSCAL or other interchange exports.
|
||||
|
||||
### Retention Index
|
||||
|
||||
Keeps compact summaries over time while allowing raw artifact retention to be
|
||||
bounded by policy.
|
||||
|
||||
## Extension Archetypes
|
||||
|
||||
### Executable Harness Extension
|
||||
|
||||
Runs an external TCK, CTS, or conformance suite.
|
||||
|
||||
Examples: `open-cmis-tck`, OGC TEAM Engine, Jakarta EE TCK, Khronos CTS.
|
||||
|
||||
### Validator Extension
|
||||
|
||||
Validates structured artifacts against schemas, profiles, or data-stream
|
||||
requirements.
|
||||
|
||||
Examples: SCAP content validation, FHIR resource validation.
|
||||
|
||||
### Protocol Service Extension
|
||||
|
||||
Coordinates with an external authority-operated service.
|
||||
|
||||
Example: NIST ACVP.
|
||||
|
||||
### Hosted Suite Extension
|
||||
|
||||
Uses a hosted or locally containerized suite with named test plans.
|
||||
|
||||
Examples: OpenID Conformance Suite, Inferno.
|
||||
|
||||
### Repository Quality Extension
|
||||
|
||||
Runs checks against repository configuration, development process, supply chain
|
||||
signals, and release hygiene.
|
||||
|
||||
Example: OpenSSF Scorecard.
|
||||
|
||||
### Procedural Evidence Extension
|
||||
|
||||
Guides collection of policy, process, and control evidence where no official
|
||||
executable harness exists.
|
||||
|
||||
Examples: GDPR, SOC 2, HIPAA, NF Z 42-013, NF 461, ISO 14641, ISO 15489.
|
||||
|
||||
### Hybrid Extension
|
||||
|
||||
Combines automated checks, manual evidence, external auditor review, and imported
|
||||
result packages.
|
||||
|
||||
## Core Data Contracts
|
||||
|
||||
The first implementation should define these as simple JSON/YAML schemas before
|
||||
building complex runtime code.
|
||||
|
||||
### `Authority`
|
||||
|
||||
- `id`
|
||||
- `name`
|
||||
- `authority_type`
|
||||
- `source_urls`
|
||||
- `frameworks`
|
||||
- `license_posture`
|
||||
- `access_constraints`
|
||||
- `certification_boundary`
|
||||
- `lifecycle_status`
|
||||
|
||||
### `ExtensionManifest`
|
||||
|
||||
- `id`
|
||||
- `name`
|
||||
- `version`
|
||||
- `extension_type`
|
||||
- `supported_frameworks`
|
||||
- `profile_schemas`
|
||||
- `check_groups`
|
||||
- `runner_entrypoints`
|
||||
- `normalizers`
|
||||
- `mappings`
|
||||
- `report_fragments`
|
||||
- `dependencies`
|
||||
- `restricted_assets`
|
||||
|
||||
### `Framework`
|
||||
|
||||
- `id`
|
||||
- `authority_id`
|
||||
- `name`
|
||||
- `version`
|
||||
- `status`
|
||||
- `source_urls`
|
||||
- `requirement_index`
|
||||
- `profile_index`
|
||||
- `license_posture`
|
||||
|
||||
### `TargetProfile`
|
||||
|
||||
- `id`
|
||||
- `subject_type`
|
||||
- `subject_name`
|
||||
- `environment`
|
||||
- `scope`
|
||||
- `endpoints`
|
||||
- `artifacts`
|
||||
- `credentials_ref`
|
||||
- `declared_capabilities`
|
||||
- `known_gaps`
|
||||
|
||||
### `AssessmentProfile`
|
||||
|
||||
- `id`
|
||||
- `framework_refs`
|
||||
- `extension_refs`
|
||||
- `target_profile_ref`
|
||||
- `selected_check_groups`
|
||||
- `expectations_ref`
|
||||
- `waivers_ref`
|
||||
- `output_policy`
|
||||
- `retention_policy`
|
||||
|
||||
### `CheckDefinition`
|
||||
|
||||
- `id`
|
||||
- `extension_id`
|
||||
- `check_type`
|
||||
- `framework_refs`
|
||||
- `requirement_refs`
|
||||
- `inputs`
|
||||
- `preconditions`
|
||||
- `timeout`
|
||||
- `runner_ref`
|
||||
- `expected_artifacts`
|
||||
|
||||
### `RunPlan`
|
||||
|
||||
- `id`
|
||||
- `assessment_profile_snapshot`
|
||||
- `extension_snapshots`
|
||||
- `source_lock`
|
||||
- `ordered_steps`
|
||||
- `credential_refs`
|
||||
- `artifact_policy`
|
||||
- `runtime_policy`
|
||||
|
||||
### `RawArtifact`
|
||||
|
||||
- `id`
|
||||
- `run_id`
|
||||
- `path`
|
||||
- `media_type`
|
||||
- `producer`
|
||||
- `checksum`
|
||||
- `created_at`
|
||||
- `retention_class`
|
||||
|
||||
### `EvidenceItem`
|
||||
|
||||
- `id`
|
||||
- `run_id`
|
||||
- `extension_id`
|
||||
- `check_id`
|
||||
- `subject_ref`
|
||||
- `result`
|
||||
- `observations`
|
||||
- `facts`
|
||||
- `requirement_refs`
|
||||
- `artifact_refs`
|
||||
- `started_at`
|
||||
- `completed_at`
|
||||
|
||||
### `Finding`
|
||||
|
||||
- `id`
|
||||
- `run_id`
|
||||
- `status`
|
||||
- `severity`
|
||||
- `classification`
|
||||
- `requirement_refs`
|
||||
- `evidence_refs`
|
||||
- `expected`
|
||||
- `waiver_ref`
|
||||
- `remediation`
|
||||
|
||||
### `Waiver`
|
||||
|
||||
- `id`
|
||||
- `scope`
|
||||
- `requirement_refs`
|
||||
- `reason`
|
||||
- `owner`
|
||||
- `approved_by`
|
||||
- `created_at`
|
||||
- `expires_at`
|
||||
- `review_status`
|
||||
|
||||
### `AssessmentPackage`
|
||||
|
||||
- `id`
|
||||
- `run_id`
|
||||
- `target`
|
||||
- `frameworks`
|
||||
- `extensions`
|
||||
- `source_lock`
|
||||
- `summary`
|
||||
- `findings`
|
||||
- `evidence_refs`
|
||||
- `artifact_manifest`
|
||||
- `waivers`
|
||||
- `certification_boundary`
|
||||
- `created_at`
|
||||
|
||||
## Result Vocabulary
|
||||
|
||||
The evidence model should allow these statuses:
|
||||
|
||||
- `pass`
|
||||
- `fail`
|
||||
- `warning`
|
||||
- `manual`
|
||||
- `not_applicable`
|
||||
- `skipped`
|
||||
- `expected_gap`
|
||||
- `waiver_applied`
|
||||
- `unsupported_by_design`
|
||||
- `infrastructure_error`
|
||||
- `blocked`
|
||||
- `unknown`
|
||||
|
||||
The reporting layer should distinguish at least:
|
||||
|
||||
- conformant evidence,
|
||||
- nonconformant evidence,
|
||||
- expected limitation,
|
||||
- waived limitation,
|
||||
- missing evidence,
|
||||
- infrastructure failure,
|
||||
- human review required.
|
||||
|
||||
## Proposed Repository Layout
|
||||
|
||||
```text
|
||||
guide-board/
|
||||
INTENT.md
|
||||
README.md
|
||||
docs/
|
||||
ARCHITECTURE-BLUEPRINT.md
|
||||
schemas/
|
||||
extensions/
|
||||
CANDIDATES.md
|
||||
_template/
|
||||
open-cmis-tck/
|
||||
INTENT.md
|
||||
extension.yaml
|
||||
docs/
|
||||
schemas/
|
||||
checks/
|
||||
mappings/
|
||||
profiles/
|
||||
runners/
|
||||
normalizers/
|
||||
reports/
|
||||
workplans/
|
||||
runs/
|
||||
reports/
|
||||
workplans/
|
||||
```
|
||||
|
||||
`runs/` and `reports/` should be local generated outputs and ignored by default.
|
||||
|
||||
## Execution Flow
|
||||
|
||||
```text
|
||||
discover extensions
|
||||
-> load authority catalog
|
||||
-> validate target profile
|
||||
-> validate assessment profile
|
||||
-> plan run
|
||||
-> run preflight
|
||||
-> execute checks
|
||||
-> collect artifacts
|
||||
-> normalize evidence
|
||||
-> map findings
|
||||
-> apply expectations and waivers
|
||||
-> build assessment package
|
||||
-> write reports
|
||||
-> retain summaries
|
||||
```
|
||||
|
||||
## Run Directory Contract
|
||||
|
||||
Each run should be reproducible from captured metadata where possible.
|
||||
|
||||
```text
|
||||
runs/<run-id>/
|
||||
run.json
|
||||
plan.json
|
||||
sources.lock.json
|
||||
target-profile.snapshot.json
|
||||
assessment-profile.snapshot.json
|
||||
artifacts/
|
||||
normalized/
|
||||
evidence.json
|
||||
findings.json
|
||||
mappings.json
|
||||
reports/
|
||||
report.md
|
||||
assessment-package.json
|
||||
exports/
|
||||
```
|
||||
|
||||
## Container And Service Model
|
||||
|
||||
The local CLI should come first. Containerization should preserve the same CLI
|
||||
contracts.
|
||||
|
||||
Recommended container model:
|
||||
|
||||
- `guide-board-core` image contains the core CLI and schema tooling.
|
||||
- Extension dependencies are either installed by extension-specific images or
|
||||
mounted as external assets.
|
||||
- Profiles, credentials, runs, and reports are mounted explicitly.
|
||||
- Restricted tools are mounted from licensed local paths.
|
||||
- Network access is declared per extension and per assessment profile.
|
||||
|
||||
Optional service model:
|
||||
|
||||
- service lists extensions and profiles,
|
||||
- service validates and plans runs,
|
||||
- service starts jobs that call the CLI contracts,
|
||||
- service streams status and exposes reports,
|
||||
- service does not invent separate execution semantics.
|
||||
|
||||
Candidate API resources:
|
||||
|
||||
- `GET /extensions`
|
||||
- `GET /authorities`
|
||||
- `POST /profiles/validate`
|
||||
- `POST /assessments/plan`
|
||||
- `POST /runs`
|
||||
- `GET /runs/{run_id}`
|
||||
- `GET /runs/{run_id}/artifacts`
|
||||
- `GET /runs/{run_id}/reports`
|
||||
|
||||
## Governance Model
|
||||
|
||||
### Extension Lifecycle
|
||||
|
||||
- `candidate`: researched and registered.
|
||||
- `incubating`: has an intent and workplan.
|
||||
- `active`: runnable through core contracts.
|
||||
- `external`: maintained outside the repo but compatible.
|
||||
- `deprecated`: retained for historical runs only.
|
||||
|
||||
### Challenge And Exclusion Handling
|
||||
|
||||
Use separate concepts:
|
||||
|
||||
- authority exclusion: imported from an official TCK or program process,
|
||||
- extension challenge: local claim that a check is invalid or mis-mapped,
|
||||
- target expectation: declared optional or unsupported behavior,
|
||||
- waiver: approved and time-bounded exception,
|
||||
- defect: unexpected product or process failure.
|
||||
|
||||
The report must make these visible separately.
|
||||
|
||||
### Source Locking
|
||||
|
||||
Each run should lock:
|
||||
|
||||
- extension version,
|
||||
- framework version,
|
||||
- harness version,
|
||||
- authority source URLs,
|
||||
- test suite IDs,
|
||||
- mapping version,
|
||||
- target profile snapshot,
|
||||
- waiver snapshot.
|
||||
|
||||
## Implementation Sequence
|
||||
|
||||
1. Create schema drafts for the core data contracts.
|
||||
2. Add an extension manifest format and a minimal sample extension.
|
||||
3. Build the CLI commands: `extensions list`, `profile validate`, `plan`, `run`,
|
||||
and `report`.
|
||||
4. Integrate `open-cmis-tck` through the same contracts.
|
||||
5. Add generated-output ignores for `runs/` and `reports/`.
|
||||
6. Add container design after the CLI baseline is stable.
|
||||
7. Add optional service API around the CLI job model.
|
||||
8. Add OSCAL export and procedural evidence-pack support after the internal
|
||||
evidence model proves itself with executable extensions.
|
||||
@@ -150,6 +150,63 @@ implementation; it marks a useful pattern for later inclusion.
|
||||
- Source:
|
||||
- [OPC UA Compliance Test Tool](https://opcfoundation.org/developer-tools/certification-test-tools/opc-ua-compliance-test-tool-uactt/)
|
||||
|
||||
### `nist-scap-openscap`
|
||||
|
||||
- Status: candidate.
|
||||
- Domain: security configuration, vulnerability, patch, and technical control
|
||||
compliance automation.
|
||||
- Authority and sources: NIST SCAP, OpenSCAP ecosystem.
|
||||
- Harness pattern: machine-readable policy content, scanner/validator execution,
|
||||
profiles, local system assessment, structured results, and possible tailored
|
||||
content.
|
||||
- Why it matters: it is a strong precedent for content-driven compliance checks
|
||||
where the policy content and scanner are separate but interoperable.
|
||||
- Sources:
|
||||
- [NIST SCAP](https://csrc.nist.gov/Projects/Security-Content-Automation-Protocol)
|
||||
- [NIST SCAP 1.3](https://csrc.nist.gov/projects/security-content-automation-protocol/scap-releases/scap-1-3)
|
||||
- [OpenSCAP](https://www.open-scap.org/)
|
||||
|
||||
### `nist-oscal`
|
||||
|
||||
- Status: core-export candidate.
|
||||
- Domain: machine-readable control, implementation, assessment, and remediation
|
||||
data.
|
||||
- Authority and sources: National Institute of Standards and Technology.
|
||||
- Harness pattern: not a test harness itself; a structured interchange model for
|
||||
catalogs, profiles, system security plans, assessment plans, assessment
|
||||
results, and POA&M data.
|
||||
- Why it matters: it provides the closest official model for later exporting
|
||||
guide-board compliance evidence into assessment packages.
|
||||
- Source:
|
||||
- [NIST OSCAL Layers and Models](https://pages.nist.gov/OSCAL/learn/concepts/layer/)
|
||||
|
||||
### `cis-cat-pro`
|
||||
|
||||
- Status: candidate with access restrictions.
|
||||
- Domain: secure configuration assessment against CIS Benchmarks and CIS
|
||||
Controls.
|
||||
- Authority and sources: Center for Internet Security.
|
||||
- Harness pattern: member-access assessment tool, benchmark profiles, automated
|
||||
and manual assessment, reports mapped to CIS Controls.
|
||||
- Why it matters: it shows how guide-board should model licensed benchmark
|
||||
content and restricted tools without redistributing them.
|
||||
- Sources:
|
||||
- [CIS-CAT Pro Assessor](https://www.cisecurity.org/cybersecurity-tools/cis-cat-pro)
|
||||
- [CIS-CAT Pro Coverage Guide](https://ciscat-assessor.docs.cisecurity.org/en/latest/Coverage%20Guide/)
|
||||
|
||||
### `openssf-scorecard`
|
||||
|
||||
- Status: candidate.
|
||||
- Domain: repository security posture and software supply-chain quality.
|
||||
- Authority and sources: Open Source Security Foundation.
|
||||
- Harness pattern: automated repository checks, per-check score and risk level,
|
||||
aggregate score, remediation guidance, CI/API integration.
|
||||
- Why it matters: guide-board should support repository quality management as an
|
||||
extension family alongside formal standards and certification preparation.
|
||||
- Sources:
|
||||
- [OpenSSF Scorecard](https://openssf.org/projects/scorecard/)
|
||||
- [Scorecard documentation](https://github.com/ossf/scorecard)
|
||||
|
||||
## Non-Harness Evidence Packs
|
||||
|
||||
Some important frameworks may not have an official executable test harness in the
|
||||
@@ -182,4 +239,5 @@ The candidates point to the same core abstractions:
|
||||
- conformance class, capability, control, or requirement mapping,
|
||||
- expectation and waiver model for optional or unsupported behavior,
|
||||
- result package suitable for human review and possible certification submission,
|
||||
- explicit boundary between preparation evidence and certification decision.
|
||||
- explicit boundary between preparation evidence and certification decision,
|
||||
- optional export into formal assessment interchange formats such as OSCAL.
|
||||
|
||||
@@ -121,13 +121,33 @@ Acceptance:
|
||||
harness candidates.
|
||||
- Candidates include CMIS/OpenCMIS, OGC TEAM Engine, OpenID Foundation
|
||||
Conformance Suite, CNCF Kubernetes Conformance, web-platform-tests, Khronos
|
||||
CTS, NIST ACVP, ONC/HL7 FHIR Inferno, Jakarta EE TCK, and OPC UA CTT.
|
||||
CTS, NIST ACVP, ONC/HL7 FHIR Inferno, Jakarta EE TCK, OPC UA CTT, NIST
|
||||
SCAP/OpenSCAP, NIST OSCAL, CIS-CAT Pro, and OpenSSF Scorecard.
|
||||
- Candidate notes capture authority, harness pattern, value, and access
|
||||
constraints.
|
||||
- Non-harness compliance packs are separated from executable conformance harness
|
||||
candidates.
|
||||
|
||||
## D1.4 - Core Contract Schemas
|
||||
## D1.4 - Core Architecture Blueprint
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T004A
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "503cb054-e8a7-42e6-a171-e57c7188d835"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `docs/ARCHITECTURE-BLUEPRINT.md` captures core concepts, precedent lessons,
|
||||
component boundaries, extension archetypes, execution flow, run directory
|
||||
contract, and governance model.
|
||||
- The blueprint distinguishes executable harnesses, validators,
|
||||
protocol-driven services, hosted suites, repository quality scanners, and
|
||||
procedural evidence collectors.
|
||||
- The blueprint names the next schema and CLI implementation sequence.
|
||||
|
||||
## D1.5 - Core Contract Schemas
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T004
|
||||
@@ -146,7 +166,7 @@ Acceptance:
|
||||
- Schemas include source URL, source version, harness version, license/access
|
||||
posture, and certification boundary fields.
|
||||
|
||||
## D1.5 - Local CLI Baseline
|
||||
## D1.6 - Local CLI Baseline
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T005
|
||||
@@ -163,7 +183,7 @@ Acceptance:
|
||||
- CLI can execute a no-op/sample extension to prove core contracts independent
|
||||
of CMIS.
|
||||
|
||||
## D1.6 - Extension SDK Skeleton
|
||||
## D1.7 - Extension SDK Skeleton
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T006
|
||||
@@ -180,7 +200,7 @@ Acceptance:
|
||||
- Extension ownership boundaries make later extraction to a separate repository
|
||||
straightforward.
|
||||
|
||||
## D1.7 - CMIS Seed Extension Integration
|
||||
## D1.8 - CMIS Seed Extension Integration
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T007
|
||||
@@ -196,7 +216,7 @@ Acceptance:
|
||||
- CMIS output normalizes into the same evidence model used by other extensions.
|
||||
- CMIS capability mappings are extension-owned.
|
||||
|
||||
## D1.8 - Containerized Execution Design
|
||||
## D1.9 - Containerized Execution Design
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T008
|
||||
@@ -213,7 +233,7 @@ Acceptance:
|
||||
- Restricted or license-gated harnesses are represented as mounted external
|
||||
assets, not redistributed guide-board content.
|
||||
|
||||
## D1.9 - Optional Local Service API
|
||||
## D1.10 - Optional Local Service API
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T009
|
||||
@@ -229,7 +249,7 @@ Acceptance:
|
||||
- Long-running jobs are tracked without blocking the API process.
|
||||
- CLI remains the source of truth for execution semantics.
|
||||
|
||||
## D1.10 - Compliance Evidence Pack Strategy
|
||||
## D1.11 - Compliance Evidence Pack Strategy
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T010
|
||||
@@ -253,6 +273,7 @@ Acceptance:
|
||||
- CMIS is represented as the first extension, not the root product.
|
||||
- The root architecture is broad enough for official conformance harnesses and
|
||||
procedural evidence packs.
|
||||
- The root architecture blueprint is documented and linked from README.
|
||||
- At least one extension can be run through local CLI contracts.
|
||||
- Candidate extensions are registered with authority, source, access, and
|
||||
architecture notes.
|
||||
|
||||
Reference in New Issue
Block a user