generated from coulomb/repo-seed
Restructured as guide-board compliance framework
This commit is contained in:
2
.gitignore
vendored
2
.gitignore
vendored
@@ -174,3 +174,5 @@ cython_debug/
|
||||
# PyPI configuration file
|
||||
.pypirc
|
||||
|
||||
# Editor swap files
|
||||
*.swp
|
||||
|
||||
248
INTENT.md
248
INTENT.md
@@ -2,146 +2,182 @@
|
||||
|
||||
## Project Name
|
||||
|
||||
`open-cmis-tck`
|
||||
`guide-board`
|
||||
|
||||
## Purpose
|
||||
|
||||
`open-cmis-tck` provides a reusable CMIS compatibility test facility as a
|
||||
service. Its first target is `kontextual-engine`, but the repository should be
|
||||
useful for any CMIS-like repository that wants repeatable capability evidence
|
||||
without embedding a Java/Maven TCK harness into the product repository itself.
|
||||
`guide-board` is a certification and compliance preparation framework. It helps
|
||||
teams turn claims about standards support, regulatory posture, and repository
|
||||
quality into structured evidence that can be reviewed, repeated, compared, and
|
||||
used during assessments.
|
||||
|
||||
The project turns CMIS capability claims into evidence:
|
||||
The project provides the core evidence model, orchestration contracts, extension
|
||||
layout, and reporting surface. Domain-specific standards, regulations, and
|
||||
conformance tools live in extensions. An extension may wrap an executable test
|
||||
harness, validate machine-readable artifacts, guide procedural evidence
|
||||
collection, or combine several of those approaches.
|
||||
|
||||
- execute selected Apache Chemistry OpenCMIS TCK groups against a target access
|
||||
point,
|
||||
- capture results in compact machine-readable reports,
|
||||
- classify failures by CMIS capability area,
|
||||
- distinguish unsupported-by-design behavior from implementation defects,
|
||||
- produce scorecard inputs that downstream projects can use for capability
|
||||
planning.
|
||||
The first extension is `open-cmis-tck`, which packages CMIS compatibility testing
|
||||
around Apache Chemistry OpenCMIS TCK execution. It is the first concrete use case,
|
||||
not the boundary of the project.
|
||||
|
||||
`guide-board` does not issue certifications, provide audit assurance, replace
|
||||
legal counsel, or act as a certification body. It supports preparation and
|
||||
execution by making evidence, assumptions, gaps, and mappings explicit.
|
||||
|
||||
## Product Thesis
|
||||
|
||||
CMIS interoperability is most useful when it is honest. A repository does not
|
||||
need to support every optional CMIS 1.1 capability, but it must accurately
|
||||
advertise what it supports and return stable diagnostics for everything else.
|
||||
Compliance work is useful when it is evidence-driven, source-aware, and honest
|
||||
about uncertainty. A team should be able to say what it claims, which authority or
|
||||
standard that claim comes from, what evidence supports it, which gaps are known,
|
||||
which checks were not applicable, and which conclusions still require human or
|
||||
accredited review.
|
||||
|
||||
This project exists to make that honesty testable.
|
||||
`guide-board` exists to make that process repeatable.
|
||||
|
||||
## Primary Use Case
|
||||
|
||||
Given a running CMIS Browser Binding access point, such as:
|
||||
Given a target system or repository, a selected assessment profile, and one or
|
||||
more extensions, `guide-board` should:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:8000/cmis/compat-tck/browser
|
||||
```
|
||||
|
||||
the harness should:
|
||||
|
||||
1. load a target profile configuration,
|
||||
2. run selected OpenCMIS TCK checks,
|
||||
3. normalize raw TCK output,
|
||||
4. map pass/fail/skip results to capability groups,
|
||||
5. write a compact report that can be compared over time,
|
||||
6. optionally expose the run and result through a small local service API.
|
||||
1. load target and assessment profile configuration,
|
||||
2. resolve the relevant extension and authority metadata,
|
||||
3. run preflight checks before expensive or regulated checks,
|
||||
4. execute harnesses, validators, or evidence collection steps,
|
||||
5. preserve raw artifacts where useful,
|
||||
6. normalize results into a stable evidence model,
|
||||
7. map evidence to capabilities, controls, conformance classes, or requirements,
|
||||
8. distinguish pass, fail, expected gap, waiver, unsupported by design, and
|
||||
infrastructure error,
|
||||
9. write compact JSON and Markdown assessment reports,
|
||||
10. retain run summaries for comparison over time.
|
||||
|
||||
## Intended Users
|
||||
|
||||
- Engine developers validating a CMIS adapter.
|
||||
- Integration engineers checking a customer repository profile.
|
||||
- Product owners tracking CMIS readiness over time.
|
||||
- Automated agents that need structured evidence before adjusting a capability
|
||||
scorecard.
|
||||
- Engineering teams preparing technical conformance evidence.
|
||||
- Product owners tracking certification and compliance readiness.
|
||||
- Compliance and quality teams coordinating evidence across repositories.
|
||||
- Integration teams validating customer or partner systems against declared
|
||||
standards.
|
||||
- Automated agents that need structured evidence before changing scorecards,
|
||||
release gates, or repository quality posture.
|
||||
|
||||
## Core Concepts
|
||||
|
||||
- Authority: the standards body, regulator, certification program, or project
|
||||
that defines requirements or tests.
|
||||
- Framework: a named standard, regulation, profile, certification program, or
|
||||
quality policy.
|
||||
- Extension: a domain-specific package that knows how to gather and normalize
|
||||
evidence for one framework or family of frameworks.
|
||||
- Target profile: the system, repository, service, artifact, or process being
|
||||
assessed.
|
||||
- Check: an executable, validator, probe, manual evidence request, or procedural
|
||||
step.
|
||||
- Evidence: raw and normalized material produced by a check.
|
||||
- Mapping: the relationship between evidence and a capability, control,
|
||||
conformance class, requirement, or scorecard dimension.
|
||||
- Expectation: a declared posture for an optional capability, known gap, accepted
|
||||
risk, or unsupported-by-design feature.
|
||||
- Waiver: a time-bounded and reviewable exception that prevents expected gaps from
|
||||
hiding unexpected failures.
|
||||
- Assessment package: the normalized result set, human report, source metadata,
|
||||
raw artifact pointers, and decision boundary.
|
||||
|
||||
## Extension Model
|
||||
|
||||
Extensions live under `extensions/<extension-id>/` while incubating in this
|
||||
repository. Each extension should have its own `INTENT.md` so it can later become
|
||||
a separate repository without losing product boundaries.
|
||||
|
||||
An extension may provide:
|
||||
|
||||
- source and authority metadata,
|
||||
- target profile schemas,
|
||||
- harness installation notes,
|
||||
- preflight probes,
|
||||
- runners or validator adapters,
|
||||
- normalization rules,
|
||||
- control or capability mappings,
|
||||
- report fragments,
|
||||
- workplans,
|
||||
- sample profiles and fixtures.
|
||||
|
||||
The core must stay extension-neutral. It should know how to orchestrate,
|
||||
normalize, map, retain, and report evidence, but it should not embed CMIS,
|
||||
healthcare, identity, geospatial, cryptographic, privacy, or records-management
|
||||
policy directly.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- target profile configuration,
|
||||
- OpenCMIS TCK orchestration,
|
||||
- deterministic smoke probes for repository info and capability flags,
|
||||
- result normalization,
|
||||
- capability-group mapping,
|
||||
- compact JSON and Markdown reports,
|
||||
- service API for starting runs and reading results,
|
||||
- historical result retention suitable for trend and scorecard updates,
|
||||
- adapters for running against local development services.
|
||||
- extension registry and lifecycle,
|
||||
- target and assessment profile contracts,
|
||||
- authority and source metadata,
|
||||
- check orchestration,
|
||||
- local CLI-first execution,
|
||||
- optional service API for local or containerized operation,
|
||||
- normalized evidence, finding, waiver, and report schemas,
|
||||
- compact historical result retention,
|
||||
- extension adapters for official or widely used conformance harnesses,
|
||||
- procedural evidence packs for frameworks that do not have official executable
|
||||
test harnesses.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- implementing CMIS server behavior,
|
||||
- replacing Apache Chemistry/OpenCMIS,
|
||||
- claiming formal certification,
|
||||
- embedding target-project domain knowledge into the harness core,
|
||||
- requiring every consuming repository to use the same technology stack.
|
||||
- issuing certifications,
|
||||
- claiming audit assurance,
|
||||
- replacing accredited certification bodies or qualified auditors,
|
||||
- replacing legal, privacy, security, or records-management counsel,
|
||||
- redistributing proprietary standards text or restricted test suites without a
|
||||
license,
|
||||
- hiding source/version uncertainty,
|
||||
- embedding target-project domain knowledge in the core.
|
||||
|
||||
## Boundaries
|
||||
## First Extension
|
||||
|
||||
`open-cmis-tck` is a test facility, not a CMIS implementation.
|
||||
The first extension is `extensions/open-cmis-tck/`.
|
||||
|
||||
Target systems own:
|
||||
It should wrap selected Apache Chemistry OpenCMIS TCK checks against a configured
|
||||
CMIS Browser Binding endpoint, normalize the output, map results to CMIS
|
||||
capability groups, and produce guide-board-compatible evidence reports.
|
||||
|
||||
- CMIS endpoints,
|
||||
- repository data fixtures,
|
||||
- authentication and authorization behavior,
|
||||
- supported/unsupported capability decisions,
|
||||
- product scorecards.
|
||||
## Initial Candidate Families
|
||||
|
||||
This project owns:
|
||||
Initial candidate extensions are registered in `extensions/CANDIDATES.md`. They
|
||||
include official or authority-backed conformance harness patterns from OGC, the
|
||||
OpenID Foundation, CNCF Kubernetes, W3C/WHATWG web-platform-tests, Khronos CTS,
|
||||
NIST ACVP, ONC/HL7 FHIR Inferno, Jakarta EE TCK, OPC UA CTT, and CMIS/OpenCMIS.
|
||||
|
||||
- harness execution,
|
||||
- result normalization,
|
||||
- capability-group classification,
|
||||
- stable reporting contracts,
|
||||
- optional local service orchestration.
|
||||
|
||||
## First Target Integration
|
||||
|
||||
The first target profile is `kontextual-engine` `compat-tck`.
|
||||
|
||||
Expected target URL:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:8000/cmis/compat-tck/browser
|
||||
```
|
||||
|
||||
Initial expected posture:
|
||||
|
||||
- selected repository/type checks should pass,
|
||||
- selected object/content checks should pass,
|
||||
- navigation, query, ACL, and versioning checks may partially pass,
|
||||
- AtomPub and Web Services are not target bindings,
|
||||
- unsupported optional capabilities should be treated as expected skips or
|
||||
explained gaps, not failures by default.
|
||||
The point of studying these candidates is not to implement everything at once. It
|
||||
is to make the core architecture fit the real shapes of conformance work:
|
||||
profile selection, source versioning, harness setup, raw artifact retention,
|
||||
normalization, requirement mapping, challenge/waiver handling, and certification
|
||||
submission boundaries.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
The project is useful when it can:
|
||||
`guide-board` is useful when it can:
|
||||
|
||||
- run from a clean checkout with documented Java/Maven requirements,
|
||||
- verify that a target CMIS endpoint is reachable before invoking the TCK,
|
||||
- execute a selected TCK subset against a configured target,
|
||||
- produce a normalized result file,
|
||||
- map results to capability groups,
|
||||
- identify unexpected failures separately from known gaps,
|
||||
- support repeated runs without unbounded local data growth,
|
||||
- provide enough evidence to update a downstream CMIS capability scorecard.
|
||||
- run from a clean checkout with a documented local baseline,
|
||||
- register extensions without changing core code,
|
||||
- run at least one extension end to end,
|
||||
- preserve raw evidence without making raw logs the primary interface,
|
||||
- produce compact machine-readable and human-readable reports,
|
||||
- identify expected gaps separately from unexpected failures,
|
||||
- track authority names, framework versions, harness versions, and source links,
|
||||
- support later containerized execution without changing assessment contracts,
|
||||
- help teams prepare for certifications and compliance assessments without
|
||||
overstating what the tool itself can certify.
|
||||
|
||||
## Design Principles
|
||||
|
||||
- Keep target-specific policy outside the harness core.
|
||||
- Make unsupported capabilities explicit and machine-readable.
|
||||
- Prefer compact historical records over large raw logs.
|
||||
- Preserve raw TCK output as optional artifacts, not the primary interface.
|
||||
- Make local development easy before adding distributed execution.
|
||||
- Treat score changes as evidence-driven, not aspirational.
|
||||
|
||||
## Initial Milestones
|
||||
|
||||
1. Repository foundation and target profile schema.
|
||||
2. OpenCMIS TCK runner wrapper.
|
||||
3. Result normalization and capability mapping.
|
||||
4. Kontextual `compat-tck` target profile.
|
||||
5. Optional service API for local run orchestration.
|
||||
6. Scorecard export contract.
|
||||
- Keep the core small, boring, and extension-neutral.
|
||||
- Treat official source metadata as part of the evidence.
|
||||
- Make unsupported or untested areas explicit.
|
||||
- Prefer local, inspectable execution before distributed service operation.
|
||||
- Preserve raw artifacts by reference, not as the main product interface.
|
||||
- Separate evidence collection from certification conclusions.
|
||||
- Design for both executable harnesses and procedural compliance evidence.
|
||||
- Make later extension extraction to separate repositories straightforward.
|
||||
|
||||
15
README.md
15
README.md
@@ -1,10 +1,15 @@
|
||||
# open-cmis-tck
|
||||
# guide-board
|
||||
|
||||
A reusable CMIS compatibility test facility built around selected Apache
|
||||
Chemistry OpenCMIS TCK execution, result normalization, and capability scoring
|
||||
evidence.
|
||||
`guide-board` is a certification and compliance preparation framework. It turns
|
||||
standards, conformance, regulatory, and repository-quality claims into structured
|
||||
evidence that can be reviewed, repeated, compared, and used during assessments.
|
||||
|
||||
The root project owns the framework contracts. Domain-specific work lives in
|
||||
extensions.
|
||||
|
||||
See:
|
||||
|
||||
- [INTENT.md](INTENT.md)
|
||||
- [workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md](workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md)
|
||||
- [extensions/CANDIDATES.md](extensions/CANDIDATES.md)
|
||||
- [extensions/open-cmis-tck/INTENT.md](extensions/open-cmis-tck/INTENT.md)
|
||||
- [workplans/GUIDE-BOARD-WP-0001-bootstrapping.md](workplans/GUIDE-BOARD-WP-0001-bootstrapping.md)
|
||||
|
||||
185
extensions/CANDIDATES.md
Normal file
185
extensions/CANDIDATES.md
Normal file
@@ -0,0 +1,185 @@
|
||||
# Extension Candidates
|
||||
|
||||
This registry captures official or authority-backed conformance harnesses that
|
||||
can shape `guide-board` extension design. A candidate does not imply immediate
|
||||
implementation; it marks a useful pattern for later inclusion.
|
||||
|
||||
## Selection Criteria
|
||||
|
||||
- The harness, validator, or test program is maintained by a standards body,
|
||||
regulator, certification authority, foundation, or recognized upstream project.
|
||||
- It produces executable or structured evidence, not only narrative guidance.
|
||||
- It teaches a reusable architecture pattern for profiles, checks, artifacts,
|
||||
normalization, mappings, waivers, or certification boundaries.
|
||||
- Its license and access model can be represented honestly, even when the tool
|
||||
itself is restricted.
|
||||
|
||||
## Registered Candidates
|
||||
|
||||
### `open-cmis-tck`
|
||||
|
||||
- Status: seed extension.
|
||||
- Domain: CMIS repository interoperability.
|
||||
- Authority and sources: OASIS CMIS standard family, Apache Chemistry OpenCMIS
|
||||
TCK artifacts and APIs.
|
||||
- Harness pattern: Java/Maven TCK wrapper, target endpoint profile, selected test
|
||||
groups, raw logs, normalized capability evidence.
|
||||
- Notes: Apache Chemistry/OpenCMIS appears retired, so this extension must track
|
||||
maintenance status and avoid presenting TCK results as formal certification.
|
||||
- Sources:
|
||||
- [Apache Chemistry OpenCMIS TCK package](https://chemistry.apache.org/java/javadoc/org/apache/chemistry/opencmis/tck/package-summary.html)
|
||||
- [Maven Central artifact](https://central.sonatype.com/artifact/org.apache.chemistry.opencmis/chemistry-opencmis-test-tck)
|
||||
|
||||
### `ogc-team-engine`
|
||||
|
||||
- Status: high-priority candidate.
|
||||
- Domain: geospatial services, APIs, schemas, clients, and data.
|
||||
- Authority and sources: Open Geospatial Consortium Compliance Testing Program.
|
||||
- Harness pattern: multi-suite conformance engine, OGC CTL/TestNG tests, web and
|
||||
command-line execution, session storage, conformance classes, certification
|
||||
submission boundary.
|
||||
- Why it matters: it is one of the clearest examples of a general conformance
|
||||
engine plus installable executable test suites.
|
||||
- Sources:
|
||||
- [TEAM Engine documentation](https://opengeospatial.github.io/teamengine/)
|
||||
- [OGC validator](https://cite.opengeospatial.org/teamengine/)
|
||||
|
||||
### `openid-conformance-suite`
|
||||
|
||||
- Status: high-priority candidate.
|
||||
- Domain: identity, authentication, authorization, OpenID Connect, FAPI.
|
||||
- Authority and sources: OpenID Foundation.
|
||||
- Harness pattern: open source conformance suite, hosted service, local Docker
|
||||
install, test plans, public/staging environments, CI runner, certification fee
|
||||
boundary.
|
||||
- Why it matters: it cleanly separates free self-testing from formal OpenID
|
||||
certification and shows how runnable profiles become certification packages.
|
||||
- Sources:
|
||||
- [OpenID Conformance Suite](https://openid.net/certification/about-conformance-suite/)
|
||||
- [OpenID Certification](https://openid.net/certification/)
|
||||
|
||||
### `kubernetes-conformance`
|
||||
|
||||
- Status: candidate.
|
||||
- Domain: cloud-native platform API conformance.
|
||||
- Authority and sources: Cloud Native Computing Foundation.
|
||||
- Harness pattern: run the same open source conformance application used for
|
||||
certification, collect result artifacts, submit results for review.
|
||||
- Why it matters: it shows a strong public result-submission workflow and a
|
||||
versioned conformance program tied to ecosystem trust.
|
||||
- Source:
|
||||
- [CNCF Certified Kubernetes Software Conformance](https://www.cncf.io/certification/software-conformance/)
|
||||
|
||||
### `web-platform-tests`
|
||||
|
||||
- Status: candidate.
|
||||
- Domain: browser and web platform interoperability.
|
||||
- Authority and sources: web-platform-tests project across W3C, WHATWG, and web
|
||||
standards communities.
|
||||
- Harness pattern: massive shared test repository, manifest generation, local
|
||||
runner, public live deployment, public result aggregation.
|
||||
- Why it matters: it shows how specification-linked tests, shared infrastructure,
|
||||
and cross-implementation result comparison can scale.
|
||||
- Sources:
|
||||
- [web-platform-tests documentation](https://web-platform-tests.org/)
|
||||
- [web-platform-tests repository](https://github.com/web-platform-tests/wpt)
|
||||
|
||||
### `khronos-cts`
|
||||
|
||||
- Status: candidate.
|
||||
- Domain: graphics, compute, and XR API conformance.
|
||||
- Authority and sources: Khronos Group.
|
||||
- Harness pattern: conformance test suites, submission packages, explicit CTS
|
||||
versioning, automated and interactive tests, trademark/adopter boundary.
|
||||
- Why it matters: it teaches artifact packaging, conformance version metadata,
|
||||
and distinction between development testing and formal adopter conformance.
|
||||
- Sources:
|
||||
- [Vulkan CTS guide](https://docs.vulkan.org/guide/latest/vulkan_cts.html)
|
||||
- [OpenXR CTS usage guide](https://registry.khronos.org/OpenXR/conformance/cts_usage.html)
|
||||
|
||||
### `nist-acvp`
|
||||
|
||||
- Status: candidate.
|
||||
- Domain: cryptographic algorithm validation.
|
||||
- Authority and sources: National Institute of Standards and Technology.
|
||||
- Harness pattern: validation protocol, client-server vector exchange, demo and
|
||||
production environments, credentialed access, standardized result reporting.
|
||||
- Why it matters: it is a reference model for authority-operated validation where
|
||||
the external service participates directly in evidence generation.
|
||||
- Source:
|
||||
- [NIST ACVP documentation](https://pages.nist.gov/ACVP/)
|
||||
|
||||
### `hl7-fhir-inferno`
|
||||
|
||||
- Status: high-priority candidate.
|
||||
- Domain: healthcare interoperability and FHIR implementation guides.
|
||||
- Authority and sources: HL7 FHIR ecosystem and ONC Health IT Certification
|
||||
Program test tooling.
|
||||
- Harness pattern: executable test kits, implementation-guide profiles, hosted
|
||||
demonstration service, local execution, approved test-method boundary for
|
||||
specific ONC criteria.
|
||||
- Why it matters: it bridges technical API conformance and regulated health IT
|
||||
certification preparation.
|
||||
- Sources:
|
||||
- [FHIR testing framework](https://hl7.org/fhir/testing.html)
|
||||
- [Inferno on HealthIT.gov](https://fhir.healthit.gov/about/)
|
||||
- [ONC Certification API Test Kit](https://fhir.healthit.gov/suites/g10_certification)
|
||||
|
||||
### `jakarta-ee-tck`
|
||||
|
||||
- Status: candidate.
|
||||
- Domain: enterprise Java platform and specification compatibility.
|
||||
- Authority and sources: Eclipse Foundation Jakarta EE Specification Process.
|
||||
- Harness pattern: formal TCK process, compatibility rules, challenge process,
|
||||
exclusions, self-certification, license boundary.
|
||||
- Why it matters: it provides a mature process model for TCK governance, appeals,
|
||||
and controlled claims of compatibility.
|
||||
- Sources:
|
||||
- [Jakarta EE TCK Process](https://jakarta.ee/committees/specification/tckprocess/)
|
||||
- [Jakarta EE compatible implementations](https://jakarta.ee/committees/specification/compatibility/)
|
||||
|
||||
### `opc-ua-ctt`
|
||||
|
||||
- Status: candidate with access restrictions.
|
||||
- Domain: industrial automation interoperability.
|
||||
- Authority and sources: OPC Foundation.
|
||||
- Harness pattern: compliance test tool for clients and servers, profiles,
|
||||
facets, conformance units, CLI automation, member-only redistribution boundary.
|
||||
- Why it matters: it shows how guide-board should represent restricted tools
|
||||
without copying or redistributing them.
|
||||
- Source:
|
||||
- [OPC UA Compliance Test Tool](https://opcfoundation.org/developer-tools/certification-test-tools/opc-ua-compliance-test-tool-uactt/)
|
||||
|
||||
## Non-Harness Evidence Packs
|
||||
|
||||
Some important frameworks may not have an official executable test harness in the
|
||||
same sense as a TCK or conformance suite. They should still be candidates for
|
||||
guide-board evidence packs, but the extension type is different: procedural
|
||||
control evidence, policy review, artifact checks, and auditor-facing readiness
|
||||
reports.
|
||||
|
||||
Initial non-harness families to evaluate later:
|
||||
|
||||
- GDPR readiness and data protection evidence.
|
||||
- SOC 2 Trust Services Criteria evidence.
|
||||
- HIPAA privacy and security readiness evidence.
|
||||
- NF Z 42-013 and NF 461 electronic archiving evidence.
|
||||
- ISO 14641 electronic archiving evidence.
|
||||
- ISO 15489 records-management evidence.
|
||||
|
||||
These packs should cite official sources and licensed standards metadata, but
|
||||
must not redistribute proprietary standard text or imply automated certification.
|
||||
|
||||
## Architecture Lessons
|
||||
|
||||
The candidates point to the same core abstractions:
|
||||
|
||||
- authority catalog with source links, version, license, and access constraints,
|
||||
- extension manifest with harness type and execution model,
|
||||
- target profile schema per extension,
|
||||
- run plan that separates preflight, setup, execution, teardown, and reporting,
|
||||
- raw artifact store plus normalized evidence model,
|
||||
- conformance class, capability, control, or requirement mapping,
|
||||
- expectation and waiver model for optional or unsupported behavior,
|
||||
- result package suitable for human review and possible certification submission,
|
||||
- explicit boundary between preparation evidence and certification decision.
|
||||
108
extensions/open-cmis-tck/INTENT.md
Normal file
108
extensions/open-cmis-tck/INTENT.md
Normal file
@@ -0,0 +1,108 @@
|
||||
# INTENT
|
||||
|
||||
## Extension Name
|
||||
|
||||
`open-cmis-tck`
|
||||
|
||||
## Parent Project
|
||||
|
||||
`guide-board`
|
||||
|
||||
## Purpose
|
||||
|
||||
`open-cmis-tck` is the first guide-board extension. It provides a reusable CMIS
|
||||
compatibility test facility around selected Apache Chemistry OpenCMIS TCK checks.
|
||||
|
||||
The extension turns CMIS capability claims into guide-board evidence:
|
||||
|
||||
- execute selected OpenCMIS TCK groups against a target access point,
|
||||
- capture raw logs and run metadata,
|
||||
- normalize results into guide-board-compatible evidence,
|
||||
- classify outcomes by CMIS capability area,
|
||||
- distinguish unsupported-by-design behavior from implementation defects,
|
||||
- produce scorecard and assessment inputs for downstream projects.
|
||||
|
||||
## Boundary
|
||||
|
||||
This extension is a test facility, not a CMIS implementation and not a
|
||||
certification authority. It does not implement server behavior, replace the CMIS
|
||||
specification, or claim formal certification.
|
||||
|
||||
Target systems own:
|
||||
|
||||
- CMIS endpoints,
|
||||
- repository data fixtures,
|
||||
- authentication and authorization behavior,
|
||||
- supported and unsupported capability decisions,
|
||||
- product scorecards.
|
||||
|
||||
This extension owns:
|
||||
|
||||
- target profile shape for CMIS Browser Binding checks,
|
||||
- CMIS preflight probes,
|
||||
- OpenCMIS TCK orchestration,
|
||||
- CMIS result normalization,
|
||||
- CMIS capability mapping,
|
||||
- guide-board report fragments.
|
||||
|
||||
## Primary Use Case
|
||||
|
||||
Given a running CMIS Browser Binding access point, such as:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:8000/cmis/compat-tck/browser
|
||||
```
|
||||
|
||||
the extension should:
|
||||
|
||||
1. load a CMIS target profile,
|
||||
2. verify target reachability and repository posture,
|
||||
3. run selected OpenCMIS TCK checks,
|
||||
4. normalize raw TCK output,
|
||||
5. map pass, fail, skip, expected gap, and infrastructure outcomes to CMIS
|
||||
capability groups,
|
||||
6. write guide-board-compatible JSON and Markdown reports.
|
||||
|
||||
## First Target Integration
|
||||
|
||||
The first target profile is `kontextual-engine` `compat-tck`.
|
||||
|
||||
Expected target URL:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:8000/cmis/compat-tck/browser
|
||||
```
|
||||
|
||||
Initial expected posture:
|
||||
|
||||
- selected repository/type checks should pass,
|
||||
- selected object/content checks should pass,
|
||||
- navigation, query, ACL, and versioning checks may partially pass,
|
||||
- AtomPub and Web Services are not target bindings,
|
||||
- unsupported optional capabilities should be treated as expected skips or
|
||||
explained gaps, not failures by default.
|
||||
|
||||
## Source Posture
|
||||
|
||||
The extension should track source authority and maintenance status explicitly.
|
||||
Apache Chemistry OpenCMIS provides TCK package APIs and Maven artifacts, but the
|
||||
Apache Chemistry project appears retired. The extension must therefore preserve
|
||||
harness version metadata and avoid overstating the meaning of a TCK pass.
|
||||
|
||||
Useful sources:
|
||||
|
||||
- [Apache Chemistry OpenCMIS TCK package](https://chemistry.apache.org/java/javadoc/org/apache/chemistry/opencmis/tck/package-summary.html)
|
||||
- [Maven Central artifact](https://central.sonatype.com/artifact/org.apache.chemistry.opencmis/chemistry-opencmis-test-tck)
|
||||
|
||||
## Success Criteria
|
||||
|
||||
The extension is useful when it can:
|
||||
|
||||
- run from a clean checkout with documented Java/Maven requirements,
|
||||
- validate a CMIS Browser Binding target before TCK execution,
|
||||
- execute or cleanly skip selected OpenCMIS TCK groups,
|
||||
- preserve raw TCK output as optional artifacts,
|
||||
- emit normalized guide-board evidence,
|
||||
- map results to CMIS capability groups,
|
||||
- identify expected gaps separately from unexpected failures,
|
||||
- provide enough evidence to update downstream CMIS capability scorecards.
|
||||
@@ -1,15 +1,17 @@
|
||||
---
|
||||
id: OPEN-CMIS-TCK-WP-0001
|
||||
type: workplan
|
||||
type: extension-workplan
|
||||
title: "OpenCMIS TCK Harness Foundation"
|
||||
repo: open-cmis-tck
|
||||
repo: guide-board
|
||||
extension: open-cmis-tck
|
||||
domain: markitect
|
||||
status: active
|
||||
owner: codex
|
||||
planning_priority: high
|
||||
planning_order: 1
|
||||
planning_order: 2
|
||||
created: "2026-05-07"
|
||||
updated: "2026-05-07"
|
||||
superseded_root_workplan: "workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md"
|
||||
state_hub_workstream_id: "276483ee-1497-495b-8710-fd8bb7dace22"
|
||||
---
|
||||
|
||||
@@ -17,10 +19,10 @@ state_hub_workstream_id: "276483ee-1497-495b-8710-fd8bb7dace22"
|
||||
|
||||
## Purpose
|
||||
|
||||
Establish `open-cmis-tck` as a reusable CMIS compatibility test facility. The
|
||||
first usable outcome is a local harness that can run selected Apache Chemistry
|
||||
OpenCMIS TCK checks against a configured CMIS Browser Binding target and produce
|
||||
normalized capability evidence.
|
||||
Establish `open-cmis-tck` as the first guide-board extension. The first usable
|
||||
outcome is a local harness that can run selected Apache Chemistry OpenCMIS TCK
|
||||
checks against a configured CMIS Browser Binding target and produce normalized
|
||||
guide-board evidence.
|
||||
|
||||
## Background
|
||||
|
||||
@@ -29,29 +31,28 @@ with explicit unsupported flags. Its current scorecard is estimation-based.
|
||||
Running selected OpenCMIS TCK checks is the next step to turn that estimate into
|
||||
evidence-backed capability scoring.
|
||||
|
||||
This repository keeps the TCK harness separate so product repositories do not
|
||||
have to carry Java/Maven orchestration, raw TCK output, or test-facility service
|
||||
code in their own trees.
|
||||
This extension keeps CMIS-specific Java/Maven orchestration, raw TCK output, and
|
||||
capability mapping outside the guide-board core.
|
||||
|
||||
## Target Architecture
|
||||
|
||||
```text
|
||||
target profile config
|
||||
-> preflight probe
|
||||
CMIS target profile
|
||||
-> CMIS preflight probe
|
||||
-> OpenCMIS TCK runner
|
||||
-> raw run artifacts
|
||||
-> normalized result model
|
||||
-> capability-group mapper
|
||||
-> JSON/Markdown reports
|
||||
-> optional service API
|
||||
-> guide-board normalized evidence
|
||||
-> CMIS capability mapper
|
||||
-> JSON/Markdown report fragments
|
||||
```
|
||||
|
||||
## Boundary
|
||||
|
||||
This workplan builds a test facility. It does not implement CMIS server
|
||||
features, modify `kontextual-engine`, or claim formal CMIS certification.
|
||||
This workplan builds a guide-board extension. It does not implement CMIS server
|
||||
features, modify `kontextual-engine`, issue certifications, provide audit
|
||||
assurance, or replace legal, regulatory, or accredited assessment work.
|
||||
|
||||
## D1.1 - Repository Foundation
|
||||
## D1.1 - Extension Foundation
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T001
|
||||
@@ -62,12 +63,13 @@ state_hub_task_id: "add6a26d-38a8-4500-8a3e-6fdac43fee42"
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `INTENT.md` captures the project purpose, scope, boundaries, and first target.
|
||||
- Repository README identifies the project and points to the workplan.
|
||||
- `extensions/open-cmis-tck/INTENT.md` captures the extension purpose, scope,
|
||||
boundaries, and first target.
|
||||
- Extension README identifies how the extension plugs into guide-board.
|
||||
- Basic directory layout exists for configs, scripts, reports, docs, and tests.
|
||||
- Local generated artifacts are ignored where appropriate.
|
||||
|
||||
## D1.2 - Target Profile Schema
|
||||
## D1.2 - CMIS Target Profile Schema
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T002
|
||||
@@ -78,14 +80,14 @@ state_hub_task_id: "2ccc74a7-bed9-4769-8608-d579fdf3a0cd"
|
||||
|
||||
Acceptance:
|
||||
|
||||
- A target profile config format defines endpoint URL, binding, repository ID,
|
||||
credentials or auth mode, expected capability groups, known gaps, and timeout
|
||||
settings.
|
||||
- A CMIS target profile config format defines endpoint URL, binding, repository
|
||||
ID, credentials or auth mode, expected capability groups, known gaps, and
|
||||
timeout settings.
|
||||
- A `kontextual-engine` `compat-tck` example profile is included.
|
||||
- Profile validation produces actionable diagnostics for missing or invalid
|
||||
fields.
|
||||
|
||||
## D1.3 - Preflight Probe
|
||||
## D1.3 - CMIS Preflight Probe
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T003
|
||||
@@ -112,12 +114,12 @@ state_hub_task_id: "502d7586-6f9e-475e-9683-43260666d5d9"
|
||||
|
||||
Acceptance:
|
||||
|
||||
- The harness documents Java and Maven requirements.
|
||||
- The extension documents Java and Maven requirements.
|
||||
- A runner command can invoke selected OpenCMIS TCK groups against one target.
|
||||
- Raw logs and machine-readable run metadata are written under a run directory.
|
||||
- TCK execution can be skipped cleanly when Java/Maven are unavailable.
|
||||
|
||||
## D1.5 - Result Normalization
|
||||
## D1.5 - CMIS Result Normalization
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T005
|
||||
@@ -128,7 +130,7 @@ state_hub_task_id: "716486b6-6f14-41f8-8417-5015ba746005"
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Raw TCK output is normalized into a stable result schema.
|
||||
- Raw TCK output is normalized into the guide-board evidence schema.
|
||||
- Results distinguish pass, fail, expected skip, unsupported by design, and
|
||||
infrastructure error.
|
||||
- Failures include enough context to map back to TCK group, capability group,
|
||||
@@ -152,7 +154,7 @@ Acceptance:
|
||||
- Markdown reports summarize pass/fail/skip counts and unexpected gaps.
|
||||
- Known gaps do not hide unexpected failures in the same capability area.
|
||||
|
||||
## D1.7 - Optional Local Service API
|
||||
## D1.7 - Optional Local Service API Adapter
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T007
|
||||
@@ -163,9 +165,9 @@ state_hub_task_id: "a05e47bd-88db-4878-aef4-bf328790c3f0"
|
||||
|
||||
Acceptance:
|
||||
|
||||
- A small local API can list target profiles, start a run, inspect run status,
|
||||
and fetch normalized reports.
|
||||
- The service is optional; CLI operation remains the primary path.
|
||||
- The guide-board service can list CMIS profiles, start a CMIS run, inspect run
|
||||
status, and fetch normalized CMIS reports.
|
||||
- CLI operation remains the primary path.
|
||||
- Long-running TCK jobs are tracked without blocking the API process.
|
||||
|
||||
## D1.8 - Historical Result Retention
|
||||
@@ -179,17 +181,16 @@ state_hub_task_id: "c27ea43f-41ec-49d0-a890-3681455f7c6c"
|
||||
|
||||
Acceptance:
|
||||
|
||||
- The harness keeps compact run summaries over time.
|
||||
- The extension contributes compact run summaries over time.
|
||||
- Retention avoids unbounded raw log growth.
|
||||
- Summaries are suitable for trend charts and downstream capability-score
|
||||
updates.
|
||||
|
||||
## Definition Of Done
|
||||
|
||||
- A developer can configure a target CMIS Browser Binding endpoint.
|
||||
- A developer can configure a CMIS Browser Binding endpoint.
|
||||
- Preflight can verify the target before TCK execution.
|
||||
- Selected OpenCMIS TCK checks can be invoked or cleanly skipped when
|
||||
dependencies are missing.
|
||||
- Normalized JSON and Markdown reports are generated.
|
||||
- Normalized guide-board JSON and Markdown report fragments are generated.
|
||||
- The `kontextual-engine` `compat-tck` target profile is represented.
|
||||
- The repository has a clear path for StateHub registration.
|
||||
259
workplans/GUIDE-BOARD-WP-0001-bootstrapping.md
Normal file
259
workplans/GUIDE-BOARD-WP-0001-bootstrapping.md
Normal file
@@ -0,0 +1,259 @@
|
||||
---
|
||||
id: GUIDE-BOARD-WP-0001
|
||||
type: workplan
|
||||
title: "Guide Board Bootstrapping"
|
||||
repo: guide-board
|
||||
domain: markitect
|
||||
status: active
|
||||
owner: codex
|
||||
planning_priority: high
|
||||
planning_order: 1
|
||||
created: "2026-05-07"
|
||||
updated: "2026-05-07"
|
||||
supersedes:
|
||||
- "OPEN-CMIS-TCK-WP-0001"
|
||||
state_hub_workstream_id: "1d812b7d-74f0-4d71-86a7-0f390b22daf7"
|
||||
---
|
||||
|
||||
# GUIDE-BOARD-WP-0001: Guide Board Bootstrapping
|
||||
|
||||
## Purpose
|
||||
|
||||
Rename and reshape the repository from a CMIS-specific TCK harness into
|
||||
`guide-board`: a certification and compliance preparation framework with
|
||||
extension-based conformance and evidence packs.
|
||||
|
||||
The first concrete extension remains `open-cmis-tck`, but the root project now
|
||||
owns the generic architecture for profiles, checks, evidence, mappings, waivers,
|
||||
reports, retention, and eventual local/containerized operation.
|
||||
|
||||
## Background
|
||||
|
||||
The repository began as `open-cmis-tck`, a focused CMIS compatibility test
|
||||
facility. That is still valuable, but the larger product opportunity is a
|
||||
generic helper for standards enforcement, certification preparation, compliance
|
||||
readiness, and repository quality management.
|
||||
|
||||
Comparable official or authority-backed conformance programs show recurring
|
||||
patterns:
|
||||
|
||||
- a source authority and explicit framework version,
|
||||
- a target or product profile,
|
||||
- an executable harness, validator, protocol, or procedural test kit,
|
||||
- setup and preflight requirements,
|
||||
- raw artifacts and machine-readable results,
|
||||
- conformance classes, profiles, controls, or requirement mappings,
|
||||
- challenge, waiver, exclusion, or known-gap handling,
|
||||
- a clear boundary between self-testing and formal certification.
|
||||
|
||||
`guide-board` should encode those patterns once, while letting extensions provide
|
||||
domain-specific logic.
|
||||
|
||||
## Target Architecture
|
||||
|
||||
```text
|
||||
authority catalog
|
||||
-> extension registry
|
||||
-> target profile
|
||||
-> assessment profile
|
||||
-> preflight checks
|
||||
-> runner / validator / evidence collector
|
||||
-> raw artifacts
|
||||
-> normalized evidence
|
||||
-> capability / control / requirement mapping
|
||||
-> expectations and waivers
|
||||
-> assessment package
|
||||
-> reports and exports
|
||||
-> retention and trend summaries
|
||||
```
|
||||
|
||||
## Boundary
|
||||
|
||||
This project is a preparation and evidence framework. It does not issue
|
||||
certifications, provide audit assurance, replace accredited assessors, replace
|
||||
legal counsel, or redistribute restricted standards and test suites without a
|
||||
license.
|
||||
|
||||
## D1.1 - Repository Identity
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T001
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "77559888-1601-4afb-8370-495365685e22"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Root `INTENT.md` identifies the project as `guide-board`.
|
||||
- Root README introduces guide-board and points to the extension model.
|
||||
- The old CMIS-only framing no longer controls the root project.
|
||||
- Certification and audit boundaries are explicit.
|
||||
|
||||
## D1.2 - Extension Layout
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T002
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "555d6d5e-5d44-4c48-b409-b86bf5750bca"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `extensions/open-cmis-tck/INTENT.md` exists.
|
||||
- The CMIS workplan is moved under the extension.
|
||||
- Extension docs can later be extracted into a separate repository.
|
||||
- The root project remains extension-neutral.
|
||||
|
||||
## D1.3 - Candidate Harness Registry
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T003
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "35db0770-d081-464f-80a3-7fcb89efdca4"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `extensions/CANDIDATES.md` registers important official or authority-backed
|
||||
harness candidates.
|
||||
- Candidates include CMIS/OpenCMIS, OGC TEAM Engine, OpenID Foundation
|
||||
Conformance Suite, CNCF Kubernetes Conformance, web-platform-tests, Khronos
|
||||
CTS, NIST ACVP, ONC/HL7 FHIR Inferno, Jakarta EE TCK, and OPC UA CTT.
|
||||
- Candidate notes capture authority, harness pattern, value, and access
|
||||
constraints.
|
||||
- Non-harness compliance packs are separated from executable conformance harness
|
||||
candidates.
|
||||
|
||||
## D1.4 - Core Contract Schemas
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T004
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "a989702f-cc55-4751-8304-75ee2375f8ec"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Define schema drafts for authority metadata, extension manifests, target
|
||||
profiles, assessment profiles, checks, evidence, findings, waivers, mappings,
|
||||
reports, and retention summaries.
|
||||
- Schemas distinguish executable harnesses, validators, protocol-driven services,
|
||||
and procedural evidence collectors.
|
||||
- Schemas include source URL, source version, harness version, license/access
|
||||
posture, and certification boundary fields.
|
||||
|
||||
## D1.5 - Local CLI Baseline
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T005
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "f22f8cc7-27f4-4377-bb61-3e4ac2040475"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Provide a local CLI entry point for listing extensions, validating profiles,
|
||||
planning an assessment, running checks, and writing reports.
|
||||
- CLI operation works before any service API is introduced.
|
||||
- CLI can execute a no-op/sample extension to prove core contracts independent
|
||||
of CMIS.
|
||||
|
||||
## D1.6 - Extension SDK Skeleton
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T006
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "3c757929-a5e4-4c11-bbf1-6d7f26def93e"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Define how an extension declares metadata, supported frameworks, profile
|
||||
schemas, check groups, runner commands, normalizers, and report fragments.
|
||||
- Provide a minimal extension template.
|
||||
- Extension ownership boundaries make later extraction to a separate repository
|
||||
straightforward.
|
||||
|
||||
## D1.7 - CMIS Seed Extension Integration
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T007
|
||||
status: todo
|
||||
priority: high
|
||||
state_hub_task_id: "455f92b0-1d2b-43d0-aa61-464d9dc83a62"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `open-cmis-tck` runs through the guide-board core contracts rather than a
|
||||
bespoke root-level harness.
|
||||
- CMIS output normalizes into the same evidence model used by other extensions.
|
||||
- CMIS capability mappings are extension-owned.
|
||||
|
||||
## D1.8 - Containerized Execution Design
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T008
|
||||
status: todo
|
||||
priority: medium
|
||||
state_hub_task_id: "21e5c1e0-b02e-408d-a657-1771750e9b30"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Document a container model that mounts profiles, extension data, credentials,
|
||||
and output directories explicitly.
|
||||
- Runner images can include extension dependencies without polluting the core.
|
||||
- Restricted or license-gated harnesses are represented as mounted external
|
||||
assets, not redistributed guide-board content.
|
||||
|
||||
## D1.9 - Optional Local Service API
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T009
|
||||
status: todo
|
||||
priority: medium
|
||||
state_hub_task_id: "58bd6ec6-2cf3-450f-95a7-b695aaf80609"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- A local API can list extensions, validate profiles, start assessment runs,
|
||||
inspect run status, and fetch reports.
|
||||
- Long-running jobs are tracked without blocking the API process.
|
||||
- CLI remains the source of truth for execution semantics.
|
||||
|
||||
## D1.10 - Compliance Evidence Pack Strategy
|
||||
|
||||
```task
|
||||
id: GUIDE-BOARD-WP-0001-T010
|
||||
status: todo
|
||||
priority: medium
|
||||
state_hub_task_id: "2f845860-ade9-4d31-91c7-cb1c69dc4e1b"
|
||||
```
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Define how non-harness frameworks such as GDPR, SOC 2, HIPAA, NF Z 42-013,
|
||||
NF 461, ISO 14641, and ISO 15489 should be represented.
|
||||
- Separate official source metadata from internal interpretation.
|
||||
- Avoid redistributing proprietary standard text.
|
||||
- Provide a reviewable evidence-request and waiver model suitable for auditor
|
||||
collaboration.
|
||||
|
||||
## Definition Of Done
|
||||
|
||||
- The repository identity is `guide-board`.
|
||||
- CMIS is represented as the first extension, not the root product.
|
||||
- The root architecture is broad enough for official conformance harnesses and
|
||||
procedural evidence packs.
|
||||
- At least one extension can be run through local CLI contracts.
|
||||
- Candidate extensions are registered with authority, source, access, and
|
||||
architecture notes.
|
||||
- The project has a clear path from local baseline to containerized service.
|
||||
Reference in New Issue
Block a user