diff --git a/INTENT.md b/INTENT.md new file mode 100644 index 0000000..ee792ad --- /dev/null +++ b/INTENT.md @@ -0,0 +1,147 @@ +# INTENT + +## Project Name + +`open-cmis-tck` + +## Purpose + +`open-cmis-tck` provides a reusable CMIS compatibility test facility as a +service. Its first target is `kontextual-engine`, but the repository should be +useful for any CMIS-like repository that wants repeatable capability evidence +without embedding a Java/Maven TCK harness into the product repository itself. + +The project turns CMIS capability claims into evidence: + +- execute selected Apache Chemistry OpenCMIS TCK groups against a target access + point, +- capture results in compact machine-readable reports, +- classify failures by CMIS capability area, +- distinguish unsupported-by-design behavior from implementation defects, +- produce scorecard inputs that downstream projects can use for capability + planning. + +## Product Thesis + +CMIS interoperability is most useful when it is honest. A repository does not +need to support every optional CMIS 1.1 capability, but it must accurately +advertise what it supports and return stable diagnostics for everything else. + +This project exists to make that honesty testable. + +## Primary Use Case + +Given a running CMIS Browser Binding access point, such as: + +```text +http://127.0.0.1:8000/cmis/compat-tck/browser +``` + +the harness should: + +1. load a target profile configuration, +2. run selected OpenCMIS TCK checks, +3. normalize raw TCK output, +4. map pass/fail/skip results to capability groups, +5. write a compact report that can be compared over time, +6. optionally expose the run and result through a small local service API. + +## Intended Users + +- Engine developers validating a CMIS adapter. +- Integration engineers checking a customer repository profile. +- Product owners tracking CMIS readiness over time. +- Automated agents that need structured evidence before adjusting a capability + scorecard. + +## Scope + +In scope: + +- target profile configuration, +- OpenCMIS TCK orchestration, +- deterministic smoke probes for repository info and capability flags, +- result normalization, +- capability-group mapping, +- compact JSON and Markdown reports, +- service API for starting runs and reading results, +- historical result retention suitable for trend and scorecard updates, +- adapters for running against local development services. + +Out of scope: + +- implementing CMIS server behavior, +- replacing Apache Chemistry/OpenCMIS, +- claiming formal certification, +- embedding target-project domain knowledge into the harness core, +- requiring every consuming repository to use the same technology stack. + +## Boundaries + +`open-cmis-tck` is a test facility, not a CMIS implementation. + +Target systems own: + +- CMIS endpoints, +- repository data fixtures, +- authentication and authorization behavior, +- supported/unsupported capability decisions, +- product scorecards. + +This project owns: + +- harness execution, +- result normalization, +- capability-group classification, +- stable reporting contracts, +- optional local service orchestration. + +## First Target Integration + +The first target profile is `kontextual-engine` `compat-tck`. + +Expected target URL: + +```text +http://127.0.0.1:8000/cmis/compat-tck/browser +``` + +Initial expected posture: + +- selected repository/type checks should pass, +- selected object/content checks should pass, +- navigation, query, ACL, and versioning checks may partially pass, +- AtomPub and Web Services are not target bindings, +- unsupported optional capabilities should be treated as expected skips or + explained gaps, not failures by default. + +## Success Criteria + +The project is useful when it can: + +- run from a clean checkout with documented Java/Maven requirements, +- verify that a target CMIS endpoint is reachable before invoking the TCK, +- execute a selected TCK subset against a configured target, +- produce a normalized result file, +- map results to capability groups, +- identify unexpected failures separately from known gaps, +- support repeated runs without unbounded local data growth, +- provide enough evidence to update a downstream CMIS capability scorecard. + +## Design Principles + +- Keep target-specific policy outside the harness core. +- Make unsupported capabilities explicit and machine-readable. +- Prefer compact historical records over large raw logs. +- Preserve raw TCK output as optional artifacts, not the primary interface. +- Make local development easy before adding distributed execution. +- Treat score changes as evidence-driven, not aspirational. + +## Initial Milestones + +1. Repository foundation and target profile schema. +2. OpenCMIS TCK runner wrapper. +3. Result normalization and capability mapping. +4. Kontextual `compat-tck` target profile. +5. Optional service API for local run orchestration. +6. Scorecard export contract. diff --git a/README.md b/README.md index fcd7b8f..5d55892 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,10 @@ -# repo-seed +# open-cmis-tck -A git repository template to bootstrap coulomb projects from. \ No newline at end of file +A reusable CMIS compatibility test facility built around selected Apache +Chemistry OpenCMIS TCK execution, result normalization, and capability scoring +evidence. + +See: + +- [INTENT.md](INTENT.md) +- [workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md](workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md) diff --git a/workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md b/workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md new file mode 100644 index 0000000..a5a8c34 --- /dev/null +++ b/workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md @@ -0,0 +1,185 @@ +--- +id: OPEN-CMIS-TCK-WP-0001 +type: workplan +title: "OpenCMIS TCK Harness Foundation" +repo: open-cmis-tck +status: active +owner: codex +planning_priority: high +planning_order: 1 +created: "2026-05-07" +updated: "2026-05-07" +--- + +# OPEN-CMIS-TCK-WP-0001: OpenCMIS TCK Harness Foundation + +## Purpose + +Establish `open-cmis-tck` as a reusable CMIS compatibility test facility. The +first usable outcome is a local harness that can run selected Apache Chemistry +OpenCMIS TCK checks against a configured CMIS Browser Binding target and produce +normalized capability evidence. + +## Background + +`kontextual-engine` now exposes a conservative CMIS 1.1 Browser Binding subset +with explicit unsupported flags. Its current scorecard is estimation-based. +Running selected OpenCMIS TCK checks is the next step to turn that estimate into +evidence-backed capability scoring. + +This repository keeps the TCK harness separate so product repositories do not +have to carry Java/Maven orchestration, raw TCK output, or test-facility service +code in their own trees. + +## Target Architecture + +```text +target profile config + -> preflight probe + -> OpenCMIS TCK runner + -> raw run artifacts + -> normalized result model + -> capability-group mapper + -> JSON/Markdown reports + -> optional service API +``` + +## Boundary + +This workplan builds a test facility. It does not implement CMIS server +features, modify `kontextual-engine`, or claim formal CMIS certification. + +## D1.1 - Repository Foundation + +```task +id: OPEN-CMIS-TCK-WP-0001-T001 +status: todo +priority: high +``` + +Acceptance: + +- `INTENT.md` captures the project purpose, scope, boundaries, and first target. +- Repository README identifies the project and points to the workplan. +- Basic directory layout exists for configs, scripts, reports, docs, and tests. +- Local generated artifacts are ignored where appropriate. + +## D1.2 - Target Profile Schema + +```task +id: OPEN-CMIS-TCK-WP-0001-T002 +status: todo +priority: high +``` + +Acceptance: + +- A target profile config format defines endpoint URL, binding, repository ID, + credentials or auth mode, expected capability groups, known gaps, and timeout + settings. +- A `kontextual-engine` `compat-tck` example profile is included. +- Profile validation produces actionable diagnostics for missing or invalid + fields. + +## D1.3 - Preflight Probe + +```task +id: OPEN-CMIS-TCK-WP-0001-T003 +status: todo +priority: high +``` + +Acceptance: + +- Preflight checks verify target reachability, repository info, CMIS version, + binding posture, and capability flags before invoking the Java TCK. +- Unsupported optional capabilities can be accepted as expected gaps. +- Preflight output is captured as structured JSON. + +## D1.4 - OpenCMIS TCK Runner Wrapper + +```task +id: OPEN-CMIS-TCK-WP-0001-T004 +status: todo +priority: high +``` + +Acceptance: + +- The harness documents Java and Maven requirements. +- A runner command can invoke selected OpenCMIS TCK groups against one target. +- Raw logs and machine-readable run metadata are written under a run directory. +- TCK execution can be skipped cleanly when Java/Maven are unavailable. + +## D1.5 - Result Normalization + +```task +id: OPEN-CMIS-TCK-WP-0001-T005 +status: todo +priority: high +``` + +Acceptance: + +- Raw TCK output is normalized into a stable result schema. +- Results distinguish pass, fail, expected skip, unsupported by design, and + infrastructure error. +- Failures include enough context to map back to TCK group, capability group, + target profile, and raw artifact paths. + +## D1.6 - Capability Mapping And Reports + +```task +id: OPEN-CMIS-TCK-WP-0001-T006 +status: todo +priority: high +``` + +Acceptance: + +- Capability-group mapping aligns TCK groups with repository/type, navigation, + object/content, versioning, query, relationships, ACL/policy, change log, and + extension gaps. +- JSON reports are compact and suitable for scorecard ingestion. +- Markdown reports summarize pass/fail/skip counts and unexpected gaps. +- Known gaps do not hide unexpected failures in the same capability area. + +## D1.7 - Optional Local Service API + +```task +id: OPEN-CMIS-TCK-WP-0001-T007 +status: todo +priority: medium +``` + +Acceptance: + +- A small local API can list target profiles, start a run, inspect run status, + and fetch normalized reports. +- The service is optional; CLI operation remains the primary path. +- Long-running TCK jobs are tracked without blocking the API process. + +## D1.8 - Historical Result Retention + +```task +id: OPEN-CMIS-TCK-WP-0001-T008 +status: todo +priority: medium +``` + +Acceptance: + +- The harness keeps compact run summaries over time. +- Retention avoids unbounded raw log growth. +- Summaries are suitable for trend charts and downstream capability-score + updates. + +## Definition Of Done + +- A developer can configure a target CMIS Browser Binding endpoint. +- Preflight can verify the target before TCK execution. +- Selected OpenCMIS TCK checks can be invoked or cleanly skipped when + dependencies are missing. +- Normalized JSON and Markdown reports are generated. +- The `kontextual-engine` `compat-tck` target profile is represented. +- The repository has a clear path for StateHub registration.