From cf02bae73111cad18b6f1a7bd56be6e548cc8624 Mon Sep 17 00:00:00 2001 From: tegwick Date: Thu, 7 May 2026 23:07:14 +0200 Subject: [PATCH] Workplan to actually control the testrun --- ...IS-TCK-WP-0002-live-test-infrastructure.md | 252 ++++++++++++++++++ 1 file changed, 252 insertions(+) create mode 100644 workplans/OPEN-CMIS-TCK-WP-0002-live-test-infrastructure.md diff --git a/workplans/OPEN-CMIS-TCK-WP-0002-live-test-infrastructure.md b/workplans/OPEN-CMIS-TCK-WP-0002-live-test-infrastructure.md new file mode 100644 index 0000000..9dd5c3b --- /dev/null +++ b/workplans/OPEN-CMIS-TCK-WP-0002-live-test-infrastructure.md @@ -0,0 +1,252 @@ +--- +id: OPEN-CMIS-TCK-WP-0002 +type: extension-workplan +title: "Live OpenCMIS TCK Execution And Capability Maturity" +repo: open-cmis-tck +extension: open-cmis-tck +domain: markitect +status: active +owner: codex +planning_priority: high +planning_order: 3 +created: "2026-05-07" +updated: "2026-05-07" +depends_on: + - "OPEN-CMIS-TCK-WP-0001" +state_hub_workstream_id: "da3f0d16-ba8e-4147-b0fc-ab3462e0b7b0" +--- + +# OPEN-CMIS-TCK-WP-0002: Live OpenCMIS TCK Execution And Capability Maturity + +## Purpose + +Move `open-cmis-tck` from a guide-board extension foundation to a locally usable +test facility that can actually run selected Apache Chemistry OpenCMIS TCK +checks against a CMIS Browser Binding system and generate evidence about what +works, what fails, what is unsupported by design, and what capability maturity +the target currently demonstrates. + +The first delivery target is local execution on a developer workstation. Moving +the same setup to a dedicated test host or container image is deliberately +deferred until the local command path is proven. + +## Background + +WP-0001 established the extension boundary: guide-board can discover the +extension, preflight a CMIS Browser Binding endpoint, invoke a configured TCK +command, capture raw artifacts, normalize results, map requirement refs to CMIS +capability groups, and retain compact run summaries. + +What remains is the real operational layer around the Apache Chemistry +OpenCMIS TCK: + +- resolve the exact upstream TCK runtime and license/access posture, +- set up Java/Maven dependencies without vendoring restricted assets, +- map guide-board check groups to actual OpenCMIS TCK suites/classes/options, +- execute the TCK locally against a target repository, +- normalize real output, not only fixture output, +- produce a practical CMIS capability maturity scorecard. + +## Target Architecture + +```text +local workstation + -> Java/Maven/TCK dependency bootstrap + -> CMIS target profile + credential reference + -> guide-board plan + -> CMIS preflight + -> OpenCMIS TCK adapter command + -> raw TCK logs / reports / transcripts + -> normalized guide-board evidence + -> CMIS capability mapping + -> maturity scorecard + -> assessment package and Markdown report +``` + +## Boundary + +This workplan prepares and runs local conformance checks. It does not claim +formal CMIS certification, redistribute restricted third-party artifacts, +replace OASIS/Apache authority material, modify the CMIS server under test, or +move execution to a container/test host yet. + +## D2.1 - Resolve TCK Runtime And Access Model + +```task +id: OPEN-CMIS-TCK-WP-0002-T001 +status: todo +priority: high +state_hub_task_id: "f3144edb-9807-4fd6-ae29-b8db18939bc4" +``` + +Acceptance: + +- Identify the current Apache Chemistry/OpenCMIS TCK source, Maven coordinates, + executable entry point, supported Java/Maven versions, and output formats. +- Document license/access posture and whether artifacts can be downloaded, + built from source, cached locally, or must be user-provided. +- Record exact version pins or version ranges for reproducible local runs. +- Add a local dependency manifest or lock note that guide-board can snapshot in + assessment evidence. + +## D2.2 - Local Environment Bootstrap Command + +```task +id: OPEN-CMIS-TCK-WP-0002-T002 +status: todo +priority: high +state_hub_task_id: "f993c1ef-8e6f-4ad1-8375-4487887deb8b" +``` + +Acceptance: + +- Provide a single local command or script that checks Java, Maven, network/cache + access, and OpenCMIS TCK runtime availability. +- The bootstrap does not commit downloaded artifacts or secrets to the repo. +- Missing prerequisites produce actionable diagnostics. +- Successful bootstrap writes a local runtime summary that can be referenced by + later runs. + +## D2.3 - OpenCMIS TCK Adapter Invocation + +```task +id: OPEN-CMIS-TCK-WP-0002-T003 +status: todo +priority: high +state_hub_task_id: "a446a80f-fc63-4ea8-9720-9294db57ade9" +``` + +Acceptance: + +- Implement an adapter command that can invoke selected real OpenCMIS TCK checks + from guide-board `check_group` values. +- The adapter accepts Browser Binding URL, repository ID, credentials reference + or anonymous mode, timeout, output directory, and selected check group. +- Raw stdout, stderr, TCK logs, machine-readable results, and invocation + metadata are captured under the run directory. +- The adapter can run a minimal repository/type group against a live target. + +## D2.4 - Target Profiles And Credential References + +```task +id: OPEN-CMIS-TCK-WP-0002-T004 +status: todo +priority: high +state_hub_task_id: "c33a4d9a-a398-4a02-93c2-1d38e62a578f" +``` + +Acceptance: + +- Provide local target profile templates for anonymous, basic-auth, and + externally referenced credential modes without storing secrets. +- Add clear examples for a `kontextual-engine` compatibility endpoint and a + generic CMIS Browser Binding endpoint. +- Profile diagnostics confirm endpoint URL, binding, repository ID, expected + capability groups, known gaps, and runtime timeout posture. +- Documentation explains how to run a profile validation before starting a TCK + run. + +## D2.5 - Real Result Normalization + +```task +id: OPEN-CMIS-TCK-WP-0002-T005 +status: todo +priority: high +state_hub_task_id: "03ba9506-cce8-44fa-a036-2ab1ffc7a176" +``` + +Acceptance: + +- Normalize actual OpenCMIS TCK output into guide-board evidence for each + selected check group. +- Preserve real TCK case IDs, class names, failure messages, skipped tests, and + raw artifact paths. +- Distinguish pass, fail, expected gap, unsupported by design, blocked, and + infrastructure error from real output. +- Add fixtures derived from real sanitized output so normalization remains + regression-tested. + +## D2.6 - Live Pilot Run + +```task +id: OPEN-CMIS-TCK-WP-0002-T006 +status: todo +priority: high +state_hub_task_id: "d9eb9384-3352-4b71-9918-57282ee00411" +``` + +Acceptance: + +- Run the local infrastructure against at least one reachable CMIS Browser + Binding target. +- Produce a guide-board run directory with evidence, findings, mappings, + assessment package, retention summary, and Markdown report. +- The report clearly states which CMIS capability groups passed, failed, were + blocked by infrastructure, or were unsupported by design. +- Known gaps do not hide unexpected failures in the same capability group. + +## D2.7 - CMIS Capability Maturity Scorecard + +```task +id: OPEN-CMIS-TCK-WP-0002-T007 +status: todo +priority: high +state_hub_task_id: "7365052f-0d76-4fb2-b32c-236476b0f937" +``` + +Acceptance: + +- Define a small maturity model for CMIS Browser Binding capability groups. +- Score repository/type, object/content, navigation, query, relationships, + ACL/policy, versioning, change log, and extension/known-gap posture. +- Separate conformance evidence from maturity interpretation. +- Generate a compact JSON scorecard and Markdown summary suitable for downstream + product-quality tracking. + +## D2.8 - Operator Documentation + +```task +id: OPEN-CMIS-TCK-WP-0002-T008 +status: todo +priority: medium +state_hub_task_id: "da7022de-3d05-43bf-9c06-91f00c159bdc" +``` + +Acceptance: + +- Document the local setup flow from clean checkout to first live TCK run. +- Include troubleshooting for Java/Maven, TCK artifact resolution, endpoint + reachability, repository ID mismatch, credentials, timeouts, and failed tests. +- Provide copy-pasteable commands for local execution through guide-board. +- Clearly mark the formal-certification boundary. + +## D2.9 - Container/Test Host Handoff Notes + +```task +id: OPEN-CMIS-TCK-WP-0002-T009 +status: todo +priority: low +state_hub_task_id: "254d099a-5406-43b0-9497-9af594a9b911" +``` + +Acceptance: + +- Capture what would need to move into a container or dedicated test host after + local execution is stable. +- Identify mounts for profiles, credentials, cached TCK assets, and run output. +- Identify environment variables and network assumptions. +- Do not implement the container/test-host move in this workplan unless local + execution is already proven. + +## Definition Of Done + +- A developer can run one documented local bootstrap command. +- A developer can point a profile at a CMIS Browser Binding endpoint and run at + least one real OpenCMIS TCK check group through guide-board. +- The run produces normalized evidence, raw artifacts, findings, mappings, + retention summary, assessment package, and Markdown report. +- The output explains what is working, what is failing, what is blocked by + infrastructure, and what is unsupported by design. +- A first CMIS capability maturity scorecard is generated from evidence. +- Container/test-host migration is documented as a follow-on path, not a hidden + prerequisite for local use.