generated from coulomb/repo-seed
Restructured as guide-board compliance framework
This commit is contained in:
248
INTENT.md
248
INTENT.md
@@ -2,146 +2,182 @@
|
||||
|
||||
## Project Name
|
||||
|
||||
`open-cmis-tck`
|
||||
`guide-board`
|
||||
|
||||
## Purpose
|
||||
|
||||
`open-cmis-tck` provides a reusable CMIS compatibility test facility as a
|
||||
service. Its first target is `kontextual-engine`, but the repository should be
|
||||
useful for any CMIS-like repository that wants repeatable capability evidence
|
||||
without embedding a Java/Maven TCK harness into the product repository itself.
|
||||
`guide-board` is a certification and compliance preparation framework. It helps
|
||||
teams turn claims about standards support, regulatory posture, and repository
|
||||
quality into structured evidence that can be reviewed, repeated, compared, and
|
||||
used during assessments.
|
||||
|
||||
The project turns CMIS capability claims into evidence:
|
||||
The project provides the core evidence model, orchestration contracts, extension
|
||||
layout, and reporting surface. Domain-specific standards, regulations, and
|
||||
conformance tools live in extensions. An extension may wrap an executable test
|
||||
harness, validate machine-readable artifacts, guide procedural evidence
|
||||
collection, or combine several of those approaches.
|
||||
|
||||
- execute selected Apache Chemistry OpenCMIS TCK groups against a target access
|
||||
point,
|
||||
- capture results in compact machine-readable reports,
|
||||
- classify failures by CMIS capability area,
|
||||
- distinguish unsupported-by-design behavior from implementation defects,
|
||||
- produce scorecard inputs that downstream projects can use for capability
|
||||
planning.
|
||||
The first extension is `open-cmis-tck`, which packages CMIS compatibility testing
|
||||
around Apache Chemistry OpenCMIS TCK execution. It is the first concrete use case,
|
||||
not the boundary of the project.
|
||||
|
||||
`guide-board` does not issue certifications, provide audit assurance, replace
|
||||
legal counsel, or act as a certification body. It supports preparation and
|
||||
execution by making evidence, assumptions, gaps, and mappings explicit.
|
||||
|
||||
## Product Thesis
|
||||
|
||||
CMIS interoperability is most useful when it is honest. A repository does not
|
||||
need to support every optional CMIS 1.1 capability, but it must accurately
|
||||
advertise what it supports and return stable diagnostics for everything else.
|
||||
Compliance work is useful when it is evidence-driven, source-aware, and honest
|
||||
about uncertainty. A team should be able to say what it claims, which authority or
|
||||
standard that claim comes from, what evidence supports it, which gaps are known,
|
||||
which checks were not applicable, and which conclusions still require human or
|
||||
accredited review.
|
||||
|
||||
This project exists to make that honesty testable.
|
||||
`guide-board` exists to make that process repeatable.
|
||||
|
||||
## Primary Use Case
|
||||
|
||||
Given a running CMIS Browser Binding access point, such as:
|
||||
Given a target system or repository, a selected assessment profile, and one or
|
||||
more extensions, `guide-board` should:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:8000/cmis/compat-tck/browser
|
||||
```
|
||||
|
||||
the harness should:
|
||||
|
||||
1. load a target profile configuration,
|
||||
2. run selected OpenCMIS TCK checks,
|
||||
3. normalize raw TCK output,
|
||||
4. map pass/fail/skip results to capability groups,
|
||||
5. write a compact report that can be compared over time,
|
||||
6. optionally expose the run and result through a small local service API.
|
||||
1. load target and assessment profile configuration,
|
||||
2. resolve the relevant extension and authority metadata,
|
||||
3. run preflight checks before expensive or regulated checks,
|
||||
4. execute harnesses, validators, or evidence collection steps,
|
||||
5. preserve raw artifacts where useful,
|
||||
6. normalize results into a stable evidence model,
|
||||
7. map evidence to capabilities, controls, conformance classes, or requirements,
|
||||
8. distinguish pass, fail, expected gap, waiver, unsupported by design, and
|
||||
infrastructure error,
|
||||
9. write compact JSON and Markdown assessment reports,
|
||||
10. retain run summaries for comparison over time.
|
||||
|
||||
## Intended Users
|
||||
|
||||
- Engine developers validating a CMIS adapter.
|
||||
- Integration engineers checking a customer repository profile.
|
||||
- Product owners tracking CMIS readiness over time.
|
||||
- Automated agents that need structured evidence before adjusting a capability
|
||||
scorecard.
|
||||
- Engineering teams preparing technical conformance evidence.
|
||||
- Product owners tracking certification and compliance readiness.
|
||||
- Compliance and quality teams coordinating evidence across repositories.
|
||||
- Integration teams validating customer or partner systems against declared
|
||||
standards.
|
||||
- Automated agents that need structured evidence before changing scorecards,
|
||||
release gates, or repository quality posture.
|
||||
|
||||
## Core Concepts
|
||||
|
||||
- Authority: the standards body, regulator, certification program, or project
|
||||
that defines requirements or tests.
|
||||
- Framework: a named standard, regulation, profile, certification program, or
|
||||
quality policy.
|
||||
- Extension: a domain-specific package that knows how to gather and normalize
|
||||
evidence for one framework or family of frameworks.
|
||||
- Target profile: the system, repository, service, artifact, or process being
|
||||
assessed.
|
||||
- Check: an executable, validator, probe, manual evidence request, or procedural
|
||||
step.
|
||||
- Evidence: raw and normalized material produced by a check.
|
||||
- Mapping: the relationship between evidence and a capability, control,
|
||||
conformance class, requirement, or scorecard dimension.
|
||||
- Expectation: a declared posture for an optional capability, known gap, accepted
|
||||
risk, or unsupported-by-design feature.
|
||||
- Waiver: a time-bounded and reviewable exception that prevents expected gaps from
|
||||
hiding unexpected failures.
|
||||
- Assessment package: the normalized result set, human report, source metadata,
|
||||
raw artifact pointers, and decision boundary.
|
||||
|
||||
## Extension Model
|
||||
|
||||
Extensions live under `extensions/<extension-id>/` while incubating in this
|
||||
repository. Each extension should have its own `INTENT.md` so it can later become
|
||||
a separate repository without losing product boundaries.
|
||||
|
||||
An extension may provide:
|
||||
|
||||
- source and authority metadata,
|
||||
- target profile schemas,
|
||||
- harness installation notes,
|
||||
- preflight probes,
|
||||
- runners or validator adapters,
|
||||
- normalization rules,
|
||||
- control or capability mappings,
|
||||
- report fragments,
|
||||
- workplans,
|
||||
- sample profiles and fixtures.
|
||||
|
||||
The core must stay extension-neutral. It should know how to orchestrate,
|
||||
normalize, map, retain, and report evidence, but it should not embed CMIS,
|
||||
healthcare, identity, geospatial, cryptographic, privacy, or records-management
|
||||
policy directly.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- target profile configuration,
|
||||
- OpenCMIS TCK orchestration,
|
||||
- deterministic smoke probes for repository info and capability flags,
|
||||
- result normalization,
|
||||
- capability-group mapping,
|
||||
- compact JSON and Markdown reports,
|
||||
- service API for starting runs and reading results,
|
||||
- historical result retention suitable for trend and scorecard updates,
|
||||
- adapters for running against local development services.
|
||||
- extension registry and lifecycle,
|
||||
- target and assessment profile contracts,
|
||||
- authority and source metadata,
|
||||
- check orchestration,
|
||||
- local CLI-first execution,
|
||||
- optional service API for local or containerized operation,
|
||||
- normalized evidence, finding, waiver, and report schemas,
|
||||
- compact historical result retention,
|
||||
- extension adapters for official or widely used conformance harnesses,
|
||||
- procedural evidence packs for frameworks that do not have official executable
|
||||
test harnesses.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- implementing CMIS server behavior,
|
||||
- replacing Apache Chemistry/OpenCMIS,
|
||||
- claiming formal certification,
|
||||
- embedding target-project domain knowledge into the harness core,
|
||||
- requiring every consuming repository to use the same technology stack.
|
||||
- issuing certifications,
|
||||
- claiming audit assurance,
|
||||
- replacing accredited certification bodies or qualified auditors,
|
||||
- replacing legal, privacy, security, or records-management counsel,
|
||||
- redistributing proprietary standards text or restricted test suites without a
|
||||
license,
|
||||
- hiding source/version uncertainty,
|
||||
- embedding target-project domain knowledge in the core.
|
||||
|
||||
## Boundaries
|
||||
## First Extension
|
||||
|
||||
`open-cmis-tck` is a test facility, not a CMIS implementation.
|
||||
The first extension is `extensions/open-cmis-tck/`.
|
||||
|
||||
Target systems own:
|
||||
It should wrap selected Apache Chemistry OpenCMIS TCK checks against a configured
|
||||
CMIS Browser Binding endpoint, normalize the output, map results to CMIS
|
||||
capability groups, and produce guide-board-compatible evidence reports.
|
||||
|
||||
- CMIS endpoints,
|
||||
- repository data fixtures,
|
||||
- authentication and authorization behavior,
|
||||
- supported/unsupported capability decisions,
|
||||
- product scorecards.
|
||||
## Initial Candidate Families
|
||||
|
||||
This project owns:
|
||||
Initial candidate extensions are registered in `extensions/CANDIDATES.md`. They
|
||||
include official or authority-backed conformance harness patterns from OGC, the
|
||||
OpenID Foundation, CNCF Kubernetes, W3C/WHATWG web-platform-tests, Khronos CTS,
|
||||
NIST ACVP, ONC/HL7 FHIR Inferno, Jakarta EE TCK, OPC UA CTT, and CMIS/OpenCMIS.
|
||||
|
||||
- harness execution,
|
||||
- result normalization,
|
||||
- capability-group classification,
|
||||
- stable reporting contracts,
|
||||
- optional local service orchestration.
|
||||
|
||||
## First Target Integration
|
||||
|
||||
The first target profile is `kontextual-engine` `compat-tck`.
|
||||
|
||||
Expected target URL:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:8000/cmis/compat-tck/browser
|
||||
```
|
||||
|
||||
Initial expected posture:
|
||||
|
||||
- selected repository/type checks should pass,
|
||||
- selected object/content checks should pass,
|
||||
- navigation, query, ACL, and versioning checks may partially pass,
|
||||
- AtomPub and Web Services are not target bindings,
|
||||
- unsupported optional capabilities should be treated as expected skips or
|
||||
explained gaps, not failures by default.
|
||||
The point of studying these candidates is not to implement everything at once. It
|
||||
is to make the core architecture fit the real shapes of conformance work:
|
||||
profile selection, source versioning, harness setup, raw artifact retention,
|
||||
normalization, requirement mapping, challenge/waiver handling, and certification
|
||||
submission boundaries.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
The project is useful when it can:
|
||||
`guide-board` is useful when it can:
|
||||
|
||||
- run from a clean checkout with documented Java/Maven requirements,
|
||||
- verify that a target CMIS endpoint is reachable before invoking the TCK,
|
||||
- execute a selected TCK subset against a configured target,
|
||||
- produce a normalized result file,
|
||||
- map results to capability groups,
|
||||
- identify unexpected failures separately from known gaps,
|
||||
- support repeated runs without unbounded local data growth,
|
||||
- provide enough evidence to update a downstream CMIS capability scorecard.
|
||||
- run from a clean checkout with a documented local baseline,
|
||||
- register extensions without changing core code,
|
||||
- run at least one extension end to end,
|
||||
- preserve raw evidence without making raw logs the primary interface,
|
||||
- produce compact machine-readable and human-readable reports,
|
||||
- identify expected gaps separately from unexpected failures,
|
||||
- track authority names, framework versions, harness versions, and source links,
|
||||
- support later containerized execution without changing assessment contracts,
|
||||
- help teams prepare for certifications and compliance assessments without
|
||||
overstating what the tool itself can certify.
|
||||
|
||||
## Design Principles
|
||||
|
||||
- Keep target-specific policy outside the harness core.
|
||||
- Make unsupported capabilities explicit and machine-readable.
|
||||
- Prefer compact historical records over large raw logs.
|
||||
- Preserve raw TCK output as optional artifacts, not the primary interface.
|
||||
- Make local development easy before adding distributed execution.
|
||||
- Treat score changes as evidence-driven, not aspirational.
|
||||
|
||||
## Initial Milestones
|
||||
|
||||
1. Repository foundation and target profile schema.
|
||||
2. OpenCMIS TCK runner wrapper.
|
||||
3. Result normalization and capability mapping.
|
||||
4. Kontextual `compat-tck` target profile.
|
||||
5. Optional service API for local run orchestration.
|
||||
6. Scorecard export contract.
|
||||
- Keep the core small, boring, and extension-neutral.
|
||||
- Treat official source metadata as part of the evidence.
|
||||
- Make unsupported or untested areas explicit.
|
||||
- Prefer local, inspectable execution before distributed service operation.
|
||||
- Preserve raw artifacts by reference, not as the main product interface.
|
||||
- Separate evidence collection from certification conclusions.
|
||||
- Design for both executable harnesses and procedural compliance evidence.
|
||||
- Make later extension extraction to separate repositories straightforward.
|
||||
|
||||
Reference in New Issue
Block a user