Separated open-cmis-tck and guide-board repositories

extensions/CANDIDATES.md

@@ -0,0 +1,243 @@
+# Extension Candidates
+
+This registry captures official or authority-backed conformance harnesses that
+can shape `guide-board` extension design. A candidate does not imply immediate
+implementation; it marks a useful pattern for later inclusion.
+
+## Selection Criteria
+
+- The harness, validator, or test program is maintained by a standards body,
+  regulator, certification authority, foundation, or recognized upstream project.
+- It produces executable or structured evidence, not only narrative guidance.
+- It teaches a reusable architecture pattern for profiles, checks, artifacts,
+  normalization, mappings, waivers, or certification boundaries.
+- Its license and access model can be represented honestly, even when the tool
+  itself is restricted.
+
+## Registered Candidates
+
+### `open-cmis-tck`
+
+- Status: seed extension.
+- Domain: CMIS repository interoperability.
+- Authority and sources: OASIS CMIS standard family, Apache Chemistry OpenCMIS
+  TCK artifacts and APIs.
+- Harness pattern: Java/Maven TCK wrapper, target endpoint profile, selected test
+  groups, raw logs, normalized capability evidence.
+- Notes: Apache Chemistry/OpenCMIS appears retired, so this extension must track
+  maintenance status and avoid presenting TCK results as formal certification.
+- Sources:
+  - [Apache Chemistry OpenCMIS TCK package](https://chemistry.apache.org/java/javadoc/org/apache/chemistry/opencmis/tck/package-summary.html)
+  - [Maven Central artifact](https://central.sonatype.com/artifact/org.apache.chemistry.opencmis/chemistry-opencmis-test-tck)
+
+### `ogc-team-engine`
+
+- Status: high-priority candidate.
+- Domain: geospatial services, APIs, schemas, clients, and data.
+- Authority and sources: Open Geospatial Consortium Compliance Testing Program.
+- Harness pattern: multi-suite conformance engine, OGC CTL/TestNG tests, web and
+  command-line execution, session storage, conformance classes, certification
+  submission boundary.
+- Why it matters: it is one of the clearest examples of a general conformance
+  engine plus installable executable test suites.
+- Sources:
+  - [TEAM Engine documentation](https://opengeospatial.github.io/teamengine/)
+  - [OGC validator](https://cite.opengeospatial.org/teamengine/)
+
+### `openid-conformance-suite`
+
+- Status: high-priority candidate.
+- Domain: identity, authentication, authorization, OpenID Connect, FAPI.
+- Authority and sources: OpenID Foundation.
+- Harness pattern: open source conformance suite, hosted service, local Docker
+  install, test plans, public/staging environments, CI runner, certification fee
+  boundary.
+- Why it matters: it cleanly separates free self-testing from formal OpenID
+  certification and shows how runnable profiles become certification packages.
+- Sources:
+  - [OpenID Conformance Suite](https://openid.net/certification/about-conformance-suite/)
+  - [OpenID Certification](https://openid.net/certification/)
+
+### `kubernetes-conformance`
+
+- Status: candidate.
+- Domain: cloud-native platform API conformance.
+- Authority and sources: Cloud Native Computing Foundation.
+- Harness pattern: run the same open source conformance application used for
+  certification, collect result artifacts, submit results for review.
+- Why it matters: it shows a strong public result-submission workflow and a
+  versioned conformance program tied to ecosystem trust.
+- Source:
+  - [CNCF Certified Kubernetes Software Conformance](https://www.cncf.io/certification/software-conformance/)
+
+### `web-platform-tests`
+
+- Status: candidate.
+- Domain: browser and web platform interoperability.
+- Authority and sources: web-platform-tests project across W3C, WHATWG, and web
+  standards communities.
+- Harness pattern: massive shared test repository, manifest generation, local
+  runner, public live deployment, public result aggregation.
+- Why it matters: it shows how specification-linked tests, shared infrastructure,
+  and cross-implementation result comparison can scale.
+- Sources:
+  - [web-platform-tests documentation](https://web-platform-tests.org/)
+  - [web-platform-tests repository](https://github.com/web-platform-tests/wpt)
+
+### `khronos-cts`
+
+- Status: candidate.
+- Domain: graphics, compute, and XR API conformance.
+- Authority and sources: Khronos Group.
+- Harness pattern: conformance test suites, submission packages, explicit CTS
+  versioning, automated and interactive tests, trademark/adopter boundary.
+- Why it matters: it teaches artifact packaging, conformance version metadata,
+  and distinction between development testing and formal adopter conformance.
+- Sources:
+  - [Vulkan CTS guide](https://docs.vulkan.org/guide/latest/vulkan_cts.html)
+  - [OpenXR CTS usage guide](https://registry.khronos.org/OpenXR/conformance/cts_usage.html)
+
+### `nist-acvp`
+
+- Status: candidate.
+- Domain: cryptographic algorithm validation.
+- Authority and sources: National Institute of Standards and Technology.
+- Harness pattern: validation protocol, client-server vector exchange, demo and
+  production environments, credentialed access, standardized result reporting.
+- Why it matters: it is a reference model for authority-operated validation where
+  the external service participates directly in evidence generation.
+- Source:
+  - [NIST ACVP documentation](https://pages.nist.gov/ACVP/)
+
+### `hl7-fhir-inferno`
+
+- Status: high-priority candidate.
+- Domain: healthcare interoperability and FHIR implementation guides.
+- Authority and sources: HL7 FHIR ecosystem and ONC Health IT Certification
+  Program test tooling.
+- Harness pattern: executable test kits, implementation-guide profiles, hosted
+  demonstration service, local execution, approved test-method boundary for
+  specific ONC criteria.
+- Why it matters: it bridges technical API conformance and regulated health IT
+  certification preparation.
+- Sources:
+  - [FHIR testing framework](https://hl7.org/fhir/testing.html)
+  - [Inferno on HealthIT.gov](https://fhir.healthit.gov/about/)
+  - [ONC Certification API Test Kit](https://fhir.healthit.gov/suites/g10_certification)
+
+### `jakarta-ee-tck`
+
+- Status: candidate.
+- Domain: enterprise Java platform and specification compatibility.
+- Authority and sources: Eclipse Foundation Jakarta EE Specification Process.
+- Harness pattern: formal TCK process, compatibility rules, challenge process,
+  exclusions, self-certification, license boundary.
+- Why it matters: it provides a mature process model for TCK governance, appeals,
+  and controlled claims of compatibility.
+- Sources:
+  - [Jakarta EE TCK Process](https://jakarta.ee/committees/specification/tckprocess/)
+  - [Jakarta EE compatible implementations](https://jakarta.ee/committees/specification/compatibility/)
+
+### `opc-ua-ctt`
+
+- Status: candidate with access restrictions.
+- Domain: industrial automation interoperability.
+- Authority and sources: OPC Foundation.
+- Harness pattern: compliance test tool for clients and servers, profiles,
+  facets, conformance units, CLI automation, member-only redistribution boundary.
+- Why it matters: it shows how guide-board should represent restricted tools
+  without copying or redistributing them.
+- Source:
+  - [OPC UA Compliance Test Tool](https://opcfoundation.org/developer-tools/certification-test-tools/opc-ua-compliance-test-tool-uactt/)
+
+### `nist-scap-openscap`
+
+- Status: candidate.
+- Domain: security configuration, vulnerability, patch, and technical control
+  compliance automation.
+- Authority and sources: NIST SCAP, OpenSCAP ecosystem.
+- Harness pattern: machine-readable policy content, scanner/validator execution,
+  profiles, local system assessment, structured results, and possible tailored
+  content.
+- Why it matters: it is a strong precedent for content-driven compliance checks
+  where the policy content and scanner are separate but interoperable.
+- Sources:
+  - [NIST SCAP](https://csrc.nist.gov/Projects/Security-Content-Automation-Protocol)
+  - [NIST SCAP 1.3](https://csrc.nist.gov/projects/security-content-automation-protocol/scap-releases/scap-1-3)
+  - [OpenSCAP](https://www.open-scap.org/)
+
+### `nist-oscal`
+
+- Status: core-export candidate.
+- Domain: machine-readable control, implementation, assessment, and remediation
+  data.
+- Authority and sources: National Institute of Standards and Technology.
+- Harness pattern: not a test harness itself; a structured interchange model for
+  catalogs, profiles, system security plans, assessment plans, assessment
+  results, and POA&M data.
+- Why it matters: it provides the closest official model for later exporting
+  guide-board compliance evidence into assessment packages.
+- Source:
+  - [NIST OSCAL Layers and Models](https://pages.nist.gov/OSCAL/learn/concepts/layer/)
+
+### `cis-cat-pro`
+
+- Status: candidate with access restrictions.
+- Domain: secure configuration assessment against CIS Benchmarks and CIS
+  Controls.
+- Authority and sources: Center for Internet Security.
+- Harness pattern: member-access assessment tool, benchmark profiles, automated
+  and manual assessment, reports mapped to CIS Controls.
+- Why it matters: it shows how guide-board should model licensed benchmark
+  content and restricted tools without redistributing them.
+- Sources:
+  - [CIS-CAT Pro Assessor](https://www.cisecurity.org/cybersecurity-tools/cis-cat-pro)
+  - [CIS-CAT Pro Coverage Guide](https://ciscat-assessor.docs.cisecurity.org/en/latest/Coverage%20Guide/)
+
+### `openssf-scorecard`
+
+- Status: candidate.
+- Domain: repository security posture and software supply-chain quality.
+- Authority and sources: Open Source Security Foundation.
+- Harness pattern: automated repository checks, per-check score and risk level,
+  aggregate score, remediation guidance, CI/API integration.
+- Why it matters: guide-board should support repository quality management as an
+  extension family alongside formal standards and certification preparation.
+- Sources:
+  - [OpenSSF Scorecard](https://openssf.org/projects/scorecard/)
+  - [Scorecard documentation](https://github.com/ossf/scorecard)
+
+## Non-Harness Evidence Packs
+
+Some important frameworks may not have an official executable test harness in the
+same sense as a TCK or conformance suite. They should still be candidates for
+guide-board evidence packs, but the extension type is different: procedural
+control evidence, policy review, artifact checks, and auditor-facing readiness
+reports.
+
+Initial non-harness families to evaluate later:
+
+- GDPR readiness and data protection evidence.
+- SOC 2 Trust Services Criteria evidence.
+- HIPAA privacy and security readiness evidence.
+- NF Z 42-013 and NF 461 electronic archiving evidence.
+- ISO 14641 electronic archiving evidence.
+- ISO 15489 records-management evidence.
+
+These packs should cite official sources and licensed standards metadata, but
+must not redistribute proprietary standard text or imply automated certification.
+
+## Architecture Lessons
+
+The candidates point to the same core abstractions:
+
+- authority catalog with source links, version, license, and access constraints,
+- extension manifest with harness type and execution model,
+- target profile schema per extension,
+- run plan that separates preflight, setup, execution, teardown, and reporting,
+- raw artifact store plus normalized evidence model,
+- conformance class, capability, control, or requirement mapping,
+- expectation and waiver model for optional or unsupported behavior,
+- result package suitable for human review and possible certification submission,
+- explicit boundary between preparation evidence and certification decision,
+- optional export into formal assessment interchange formats such as OSCAL.

extensions/_template/extension.json

@@ -0,0 +1,33 @@
+{
+  "id": "replace-with-extension-id",
+  "name": "Replace With Extension Name",
+  "version": "0.1.0",
+  "extension_type": "executable_harness",
+  "lifecycle_status": "candidate",
+  "supported_frameworks": [],
+  "authorities": [],
+  "profile_schemas": [
+    "target-profile",
+    "assessment-profile"
+  ],
+  "check_groups": [],
+  "preflight_runner": null,
+  "runner_entrypoints": [
+    {
+      "id": "replace-with-runner-id",
+      "kind": "command",
+      "module_path": null,
+      "callable": null,
+      "command": [
+        "replace-with-command"
+      ],
+      "description": "Describe how this manifest-declared command produces JSON runner output."
+    }
+  ],
+  "normalizers": [],
+  "mappings": [],
+  "report_fragments": [],
+  "dependencies": [],
+  "restricted_assets": [],
+  "certification_boundary": "This template is not an assessment or certification authority."
+}

extensions/sample-noop/INTENT.md

@@ -0,0 +1,14 @@
+# INTENT
+
+## Extension Name
+
+`sample-noop`
+
+## Purpose
+
+`sample-noop` is a tiny guide-board extension used to prove that the core can
+discover extensions, validate profiles, and build run plans without depending on
+CMIS or any external test harness.
+
+It should stay boring. Its job is to exercise the guide-board contracts before
+real extension adapters add domain-specific runners and normalizers.

extensions/sample-noop/extension.json

@@ -0,0 +1,38 @@
+{
+  "id": "sample-noop",
+  "name": "Sample No-Op Extension",
+  "version": "0.1.0",
+  "extension_type": "procedural_evidence",
+  "lifecycle_status": "incubating",
+  "supported_frameworks": [
+    "guide-board.sample-readiness.v0"
+  ],
+  "authorities": [
+    "guide-board"
+  ],
+  "profile_schemas": [
+    "target-profile",
+    "assessment-profile"
+  ],
+  "check_groups": [
+    {
+      "id": "profile-shape",
+      "name": "Profile Shape",
+      "check_type": "manual",
+      "requirement_refs": [
+        "guide-board.sample-readiness.v0.profile-shape"
+      ],
+      "runner_ref": null
+    }
+  ],
+  "preflight_runner": null,
+  "runner_entrypoints": [],
+  "normalizers": [],
+  "mappings": [
+    "sample-readiness-map"
+  ],
+  "report_fragments": [],
+  "dependencies": [],
+  "restricted_assets": [],
+  "certification_boundary": "Development-only sample extension. It produces no certification or compliance conclusion."
+}

extensions/sample-noop/mappings/sample-readiness-map.json

@@ -0,0 +1,16 @@
+{
+  "id": "sample-readiness-map",
+  "extension_id": "sample-noop",
+  "framework_refs": [
+    "guide-board.sample-readiness.v0"
+  ],
+  "mappings": [
+    {
+      "requirement_ref": "guide-board.sample-readiness.v0.profile-shape",
+      "target_type": "quality_dimension",
+      "target_id": "profile-readiness",
+      "label": "Profile Readiness",
+      "description": "The sample target and assessment profiles can be discovered, validated, and planned."
+    }
+  ]
+}