guide-board/extensions/CANDIDATES.md

Extension Candidates

This registry captures official or authority-backed conformance harnesses that can shape guide-board extension design. A candidate does not imply immediate implementation; it marks a useful pattern for later inclusion.

Selection Criteria

• The harness, validator, or test program is maintained by a standards body, regulator, certification authority, foundation, or recognized upstream project.
• It produces executable or structured evidence, not only narrative guidance.
• It teaches a reusable architecture pattern for profiles, checks, artifacts, normalization, mappings, waivers, or certification boundaries.
• Its license and access model can be represented honestly, even when the tool itself is restricted.

Registered Candidates

open-cmis-tck

• Status: seed extension.
• Domain: CMIS repository interoperability.
• Authority and sources: OASIS CMIS standard family, Apache Chemistry OpenCMIS TCK artifacts and APIs.
• Harness pattern: Java/Maven TCK wrapper, target endpoint profile, selected test groups, raw logs, normalized capability evidence.
• Notes: Apache Chemistry/OpenCMIS appears retired, so this extension must track maintenance status and avoid presenting TCK results as formal certification.
• Sources:
  • Apache Chemistry OpenCMIS TCK package
  • Maven Central artifact

ogc-team-engine

• Status: high-priority candidate.
• Domain: geospatial services, APIs, schemas, clients, and data.
• Authority and sources: Open Geospatial Consortium Compliance Testing Program.
• Harness pattern: multi-suite conformance engine, OGC CTL/TestNG tests, web and command-line execution, session storage, conformance classes, certification submission boundary.
• Why it matters: it is one of the clearest examples of a general conformance engine plus installable executable test suites.
• Sources:
  • TEAM Engine documentation
  • OGC validator

openid-conformance-suite

• Status: high-priority candidate.
• Domain: identity, authentication, authorization, OpenID Connect, FAPI.
• Authority and sources: OpenID Foundation.
• Harness pattern: open source conformance suite, hosted service, local Docker install, test plans, public/staging environments, CI runner, certification fee boundary.
• Why it matters: it cleanly separates free self-testing from formal OpenID certification and shows how runnable profiles become certification packages.
• Sources:
  • OpenID Conformance Suite
  • OpenID Certification

kubernetes-conformance

• Status: candidate.
• Domain: cloud-native platform API conformance.
• Authority and sources: Cloud Native Computing Foundation.
• Harness pattern: run the same open source conformance application used for certification, collect result artifacts, submit results for review.
• Why it matters: it shows a strong public result-submission workflow and a versioned conformance program tied to ecosystem trust.
• Source:
  • CNCF Certified Kubernetes Software Conformance

web-platform-tests

• Status: candidate.
• Domain: browser and web platform interoperability.
• Authority and sources: web-platform-tests project across W3C, WHATWG, and web standards communities.
• Harness pattern: massive shared test repository, manifest generation, local runner, public live deployment, public result aggregation.
• Why it matters: it shows how specification-linked tests, shared infrastructure, and cross-implementation result comparison can scale.
• Sources:
  • web-platform-tests documentation
  • web-platform-tests repository

khronos-cts

• Status: candidate.
• Domain: graphics, compute, and XR API conformance.
• Authority and sources: Khronos Group.
• Harness pattern: conformance test suites, submission packages, explicit CTS versioning, automated and interactive tests, trademark/adopter boundary.
• Why it matters: it teaches artifact packaging, conformance version metadata, and distinction between development testing and formal adopter conformance.
• Sources:
  • Vulkan CTS guide
  • OpenXR CTS usage guide

nist-acvp

• Status: candidate.
• Domain: cryptographic algorithm validation.
• Authority and sources: National Institute of Standards and Technology.
• Harness pattern: validation protocol, client-server vector exchange, demo and production environments, credentialed access, standardized result reporting.
• Why it matters: it is a reference model for authority-operated validation where the external service participates directly in evidence generation.
• Source:
  • NIST ACVP documentation

hl7-fhir-inferno

• Status: high-priority candidate.
• Domain: healthcare interoperability and FHIR implementation guides.
• Authority and sources: HL7 FHIR ecosystem and ONC Health IT Certification Program test tooling.
• Harness pattern: executable test kits, implementation-guide profiles, hosted demonstration service, local execution, approved test-method boundary for specific ONC criteria.
• Why it matters: it bridges technical API conformance and regulated health IT certification preparation.
• Sources:
  • FHIR testing framework
  • Inferno on HealthIT.gov
  • ONC Certification API Test Kit

jakarta-ee-tck

• Status: candidate.
• Domain: enterprise Java platform and specification compatibility.
• Authority and sources: Eclipse Foundation Jakarta EE Specification Process.
• Harness pattern: formal TCK process, compatibility rules, challenge process, exclusions, self-certification, license boundary.
• Why it matters: it provides a mature process model for TCK governance, appeals, and controlled claims of compatibility.
• Sources:
  • Jakarta EE TCK Process
  • Jakarta EE compatible implementations

opc-ua-ctt

• Status: candidate with access restrictions.
• Domain: industrial automation interoperability.
• Authority and sources: OPC Foundation.
• Harness pattern: compliance test tool for clients and servers, profiles, facets, conformance units, CLI automation, member-only redistribution boundary.
• Why it matters: it shows how guide-board should represent restricted tools without copying or redistributing them.
• Source:
  • OPC UA Compliance Test Tool

nist-scap-openscap

• Status: candidate.
• Domain: security configuration, vulnerability, patch, and technical control compliance automation.
• Authority and sources: NIST SCAP, OpenSCAP ecosystem.
• Harness pattern: machine-readable policy content, scanner/validator execution, profiles, local system assessment, structured results, and possible tailored content.
• Why it matters: it is a strong precedent for content-driven compliance checks where the policy content and scanner are separate but interoperable.
• Sources:
  • NIST SCAP
  • NIST SCAP 1.3
  • OpenSCAP

nist-oscal

• Status: core-export candidate.
• Domain: machine-readable control, implementation, assessment, and remediation data.
• Authority and sources: National Institute of Standards and Technology.
• Harness pattern: not a test harness itself; a structured interchange model for catalogs, profiles, system security plans, assessment plans, assessment results, and POA&M data.
• Why it matters: it provides the closest official model for later exporting guide-board compliance evidence into assessment packages.
• Source:
  • NIST OSCAL Layers and Models

cis-cat-pro

• Status: candidate with access restrictions.
• Domain: secure configuration assessment against CIS Benchmarks and CIS Controls.
• Authority and sources: Center for Internet Security.
• Harness pattern: member-access assessment tool, benchmark profiles, automated and manual assessment, reports mapped to CIS Controls.
• Why it matters: it shows how guide-board should model licensed benchmark content and restricted tools without redistributing them.
• Sources:
  • CIS-CAT Pro Assessor
  • CIS-CAT Pro Coverage Guide

openssf-scorecard

• Status: candidate.
• Domain: repository security posture and software supply-chain quality.
• Authority and sources: Open Source Security Foundation.
• Harness pattern: automated repository checks, per-check score and risk level, aggregate score, remediation guidance, CI/API integration.
• Why it matters: guide-board should support repository quality management as an extension family alongside formal standards and certification preparation.
• Sources:
  • OpenSSF Scorecard
  • Scorecard documentation

Non-Harness Evidence Packs

Some important frameworks may not have an official executable test harness in the same sense as a TCK or conformance suite. They should still be candidates for guide-board evidence packs, but the extension type is different: procedural control evidence, policy review, artifact checks, and auditor-facing readiness reports.

Initial non-harness families to evaluate later:

• GDPR readiness and data protection evidence.
• SOC 2 Trust Services Criteria evidence.
• HIPAA privacy and security readiness evidence.
• NF Z 42-013 and NF 461 electronic archiving evidence.
• ISO 14641 electronic archiving evidence.
• ISO 15489 records-management evidence.

These packs should cite official sources and licensed standards metadata, but must not redistribute proprietary standard text or imply automated certification.

Architecture Lessons

The candidates point to the same core abstractions:

• authority catalog with source links, version, license, and access constraints,
• extension manifest with harness type and execution model,
• target profile schema per extension,
• run plan that separates preflight, setup, execution, teardown, and reporting,
• raw artifact store plus normalized evidence model,
• conformance class, capability, control, or requirement mapping,
• expectation and waiver model for optional or unsupported behavior,
• result package suitable for human review and possible certification submission,
• explicit boundary between preparation evidence and certification decision,
• optional export into formal assessment interchange formats such as OSCAL.