Restructured as guide-board compliance framework

This commit is contained in:
2026-05-07 09:45:11 +02:00
parent 6108f3ab51
commit 1201695ebe
7 changed files with 744 additions and 148 deletions

248
INTENT.md
View File

@@ -2,146 +2,182 @@
## Project Name
`open-cmis-tck`
`guide-board`
## Purpose
`open-cmis-tck` provides a reusable CMIS compatibility test facility as a
service. Its first target is `kontextual-engine`, but the repository should be
useful for any CMIS-like repository that wants repeatable capability evidence
without embedding a Java/Maven TCK harness into the product repository itself.
`guide-board` is a certification and compliance preparation framework. It helps
teams turn claims about standards support, regulatory posture, and repository
quality into structured evidence that can be reviewed, repeated, compared, and
used during assessments.
The project turns CMIS capability claims into evidence:
The project provides the core evidence model, orchestration contracts, extension
layout, and reporting surface. Domain-specific standards, regulations, and
conformance tools live in extensions. An extension may wrap an executable test
harness, validate machine-readable artifacts, guide procedural evidence
collection, or combine several of those approaches.
- execute selected Apache Chemistry OpenCMIS TCK groups against a target access
point,
- capture results in compact machine-readable reports,
- classify failures by CMIS capability area,
- distinguish unsupported-by-design behavior from implementation defects,
- produce scorecard inputs that downstream projects can use for capability
planning.
The first extension is `open-cmis-tck`, which packages CMIS compatibility testing
around Apache Chemistry OpenCMIS TCK execution. It is the first concrete use case,
not the boundary of the project.
`guide-board` does not issue certifications, provide audit assurance, replace
legal counsel, or act as a certification body. It supports preparation and
execution by making evidence, assumptions, gaps, and mappings explicit.
## Product Thesis
CMIS interoperability is most useful when it is honest. A repository does not
need to support every optional CMIS 1.1 capability, but it must accurately
advertise what it supports and return stable diagnostics for everything else.
Compliance work is useful when it is evidence-driven, source-aware, and honest
about uncertainty. A team should be able to say what it claims, which authority or
standard that claim comes from, what evidence supports it, which gaps are known,
which checks were not applicable, and which conclusions still require human or
accredited review.
This project exists to make that honesty testable.
`guide-board` exists to make that process repeatable.
## Primary Use Case
Given a running CMIS Browser Binding access point, such as:
Given a target system or repository, a selected assessment profile, and one or
more extensions, `guide-board` should:
```text
http://127.0.0.1:8000/cmis/compat-tck/browser
```
the harness should:
1. load a target profile configuration,
2. run selected OpenCMIS TCK checks,
3. normalize raw TCK output,
4. map pass/fail/skip results to capability groups,
5. write a compact report that can be compared over time,
6. optionally expose the run and result through a small local service API.
1. load target and assessment profile configuration,
2. resolve the relevant extension and authority metadata,
3. run preflight checks before expensive or regulated checks,
4. execute harnesses, validators, or evidence collection steps,
5. preserve raw artifacts where useful,
6. normalize results into a stable evidence model,
7. map evidence to capabilities, controls, conformance classes, or requirements,
8. distinguish pass, fail, expected gap, waiver, unsupported by design, and
infrastructure error,
9. write compact JSON and Markdown assessment reports,
10. retain run summaries for comparison over time.
## Intended Users
- Engine developers validating a CMIS adapter.
- Integration engineers checking a customer repository profile.
- Product owners tracking CMIS readiness over time.
- Automated agents that need structured evidence before adjusting a capability
scorecard.
- Engineering teams preparing technical conformance evidence.
- Product owners tracking certification and compliance readiness.
- Compliance and quality teams coordinating evidence across repositories.
- Integration teams validating customer or partner systems against declared
standards.
- Automated agents that need structured evidence before changing scorecards,
release gates, or repository quality posture.
## Core Concepts
- Authority: the standards body, regulator, certification program, or project
that defines requirements or tests.
- Framework: a named standard, regulation, profile, certification program, or
quality policy.
- Extension: a domain-specific package that knows how to gather and normalize
evidence for one framework or family of frameworks.
- Target profile: the system, repository, service, artifact, or process being
assessed.
- Check: an executable, validator, probe, manual evidence request, or procedural
step.
- Evidence: raw and normalized material produced by a check.
- Mapping: the relationship between evidence and a capability, control,
conformance class, requirement, or scorecard dimension.
- Expectation: a declared posture for an optional capability, known gap, accepted
risk, or unsupported-by-design feature.
- Waiver: a time-bounded and reviewable exception that prevents expected gaps from
hiding unexpected failures.
- Assessment package: the normalized result set, human report, source metadata,
raw artifact pointers, and decision boundary.
## Extension Model
Extensions live under `extensions/<extension-id>/` while incubating in this
repository. Each extension should have its own `INTENT.md` so it can later become
a separate repository without losing product boundaries.
An extension may provide:
- source and authority metadata,
- target profile schemas,
- harness installation notes,
- preflight probes,
- runners or validator adapters,
- normalization rules,
- control or capability mappings,
- report fragments,
- workplans,
- sample profiles and fixtures.
The core must stay extension-neutral. It should know how to orchestrate,
normalize, map, retain, and report evidence, but it should not embed CMIS,
healthcare, identity, geospatial, cryptographic, privacy, or records-management
policy directly.
## Scope
In scope:
- target profile configuration,
- OpenCMIS TCK orchestration,
- deterministic smoke probes for repository info and capability flags,
- result normalization,
- capability-group mapping,
- compact JSON and Markdown reports,
- service API for starting runs and reading results,
- historical result retention suitable for trend and scorecard updates,
- adapters for running against local development services.
- extension registry and lifecycle,
- target and assessment profile contracts,
- authority and source metadata,
- check orchestration,
- local CLI-first execution,
- optional service API for local or containerized operation,
- normalized evidence, finding, waiver, and report schemas,
- compact historical result retention,
- extension adapters for official or widely used conformance harnesses,
- procedural evidence packs for frameworks that do not have official executable
test harnesses.
Out of scope:
- implementing CMIS server behavior,
- replacing Apache Chemistry/OpenCMIS,
- claiming formal certification,
- embedding target-project domain knowledge into the harness core,
- requiring every consuming repository to use the same technology stack.
- issuing certifications,
- claiming audit assurance,
- replacing accredited certification bodies or qualified auditors,
- replacing legal, privacy, security, or records-management counsel,
- redistributing proprietary standards text or restricted test suites without a
license,
- hiding source/version uncertainty,
- embedding target-project domain knowledge in the core.
## Boundaries
## First Extension
`open-cmis-tck` is a test facility, not a CMIS implementation.
The first extension is `extensions/open-cmis-tck/`.
Target systems own:
It should wrap selected Apache Chemistry OpenCMIS TCK checks against a configured
CMIS Browser Binding endpoint, normalize the output, map results to CMIS
capability groups, and produce guide-board-compatible evidence reports.
- CMIS endpoints,
- repository data fixtures,
- authentication and authorization behavior,
- supported/unsupported capability decisions,
- product scorecards.
## Initial Candidate Families
This project owns:
Initial candidate extensions are registered in `extensions/CANDIDATES.md`. They
include official or authority-backed conformance harness patterns from OGC, the
OpenID Foundation, CNCF Kubernetes, W3C/WHATWG web-platform-tests, Khronos CTS,
NIST ACVP, ONC/HL7 FHIR Inferno, Jakarta EE TCK, OPC UA CTT, and CMIS/OpenCMIS.
- harness execution,
- result normalization,
- capability-group classification,
- stable reporting contracts,
- optional local service orchestration.
## First Target Integration
The first target profile is `kontextual-engine` `compat-tck`.
Expected target URL:
```text
http://127.0.0.1:8000/cmis/compat-tck/browser
```
Initial expected posture:
- selected repository/type checks should pass,
- selected object/content checks should pass,
- navigation, query, ACL, and versioning checks may partially pass,
- AtomPub and Web Services are not target bindings,
- unsupported optional capabilities should be treated as expected skips or
explained gaps, not failures by default.
The point of studying these candidates is not to implement everything at once. It
is to make the core architecture fit the real shapes of conformance work:
profile selection, source versioning, harness setup, raw artifact retention,
normalization, requirement mapping, challenge/waiver handling, and certification
submission boundaries.
## Success Criteria
The project is useful when it can:
`guide-board` is useful when it can:
- run from a clean checkout with documented Java/Maven requirements,
- verify that a target CMIS endpoint is reachable before invoking the TCK,
- execute a selected TCK subset against a configured target,
- produce a normalized result file,
- map results to capability groups,
- identify unexpected failures separately from known gaps,
- support repeated runs without unbounded local data growth,
- provide enough evidence to update a downstream CMIS capability scorecard.
- run from a clean checkout with a documented local baseline,
- register extensions without changing core code,
- run at least one extension end to end,
- preserve raw evidence without making raw logs the primary interface,
- produce compact machine-readable and human-readable reports,
- identify expected gaps separately from unexpected failures,
- track authority names, framework versions, harness versions, and source links,
- support later containerized execution without changing assessment contracts,
- help teams prepare for certifications and compliance assessments without
overstating what the tool itself can certify.
## Design Principles
- Keep target-specific policy outside the harness core.
- Make unsupported capabilities explicit and machine-readable.
- Prefer compact historical records over large raw logs.
- Preserve raw TCK output as optional artifacts, not the primary interface.
- Make local development easy before adding distributed execution.
- Treat score changes as evidence-driven, not aspirational.
## Initial Milestones
1. Repository foundation and target profile schema.
2. OpenCMIS TCK runner wrapper.
3. Result normalization and capability mapping.
4. Kontextual `compat-tck` target profile.
5. Optional service API for local run orchestration.
6. Scorecard export contract.
- Keep the core small, boring, and extension-neutral.
- Treat official source metadata as part of the evidence.
- Make unsupported or untested areas explicit.
- Prefer local, inspectable execution before distributed service operation.
- Preserve raw artifacts by reference, not as the main product interface.
- Separate evidence collection from certification conclusions.
- Design for both executable harnesses and procedural compliance evidence.
- Make later extension extraction to separate repositories straightforward.