open-cmis-tck/workplans/OPEN-CMIS-TCK-WP-0002-live-test-infrastructure.md

id, type, title, repo, extension, domain, status, owner, planning_priority, planning_order, created, updated, depends_on, state_hub_workstream_id

id	type	title	repo	extension	domain	status	owner	planning_priority	planning_order	created	updated	depends_on	state_hub_workstream_id
OPEN-CMIS-TCK-WP-0002	extension-workplan	Live OpenCMIS TCK Execution And Capability Maturity	open-cmis-tck	open-cmis-tck	markitect	completed	codex	high	3	2026-05-07	2026-05-14	OPEN-CMIS-TCK-WP-0001	da3f0d16-ba8e-4147-b0fc-ab3462e0b7b0

OPEN-CMIS-TCK-WP-0002: Live OpenCMIS TCK Execution And Capability Maturity

Purpose

Move open-cmis-tck from a guide-board extension foundation to a locally usable test facility that can actually run selected Apache Chemistry OpenCMIS TCK checks against a CMIS Browser Binding system and generate evidence about what works, what fails, what is unsupported by design, and what capability maturity the target currently demonstrates.

The first delivery target is local execution on a developer workstation. Moving the same setup to a dedicated test host or container image is deliberately deferred until the local command path is proven.

Background

WP-0001 established the extension boundary: guide-board can discover the extension, preflight a CMIS Browser Binding endpoint, invoke a configured TCK command, capture raw artifacts, normalize results, map requirement refs to CMIS capability groups, and retain compact run summaries.

What remains is the real operational layer around the Apache Chemistry OpenCMIS TCK:

• resolve the exact upstream TCK runtime and license/access posture,
• set up Java/Maven dependencies without vendoring restricted assets,
• map guide-board check groups to actual OpenCMIS TCK suites/classes/options,
• execute the TCK locally against a target repository,
• normalize real output, not only fixture output,
• produce a practical CMIS capability maturity scorecard.

Target Architecture

local workstation
  -> Java/Maven/TCK dependency bootstrap
  -> CMIS target profile + credential reference
  -> guide-board plan
  -> CMIS preflight
  -> OpenCMIS TCK adapter command
  -> raw TCK logs / reports / transcripts
  -> normalized guide-board evidence
  -> CMIS capability mapping
  -> maturity scorecard
  -> assessment package and Markdown report

Boundary

This workplan prepares and runs local conformance checks. It does not claim formal CMIS certification, redistribute restricted third-party artifacts, replace OASIS/Apache authority material, modify the CMIS server under test, or move execution to a container/test host yet.

D2.1 - Resolve TCK Runtime And Access Model

id: OPEN-CMIS-TCK-WP-0002-T001
status: done
priority: high
state_hub_task_id: "f3144edb-9807-4fd6-ae29-b8db18939bc4"

Acceptance:

• Identify the current Apache Chemistry/OpenCMIS TCK source, Maven coordinates, executable entry point, supported Java/Maven versions, and output formats.
• Document license/access posture and whether artifacts can be downloaded, built from source, cached locally, or must be user-provided.
• Record exact version pins or version ranges for reproducible local runs.
• Add a local dependency manifest or lock note that guide-board can snapshot in assessment evidence.

Progress:

• Resolved the first runtime pin to org.apache.chemistry.opencmis:chemistry-opencmis-test-tck:1.1.0.
• Confirmed the local entry point is org.apache.chemistry.opencmis.tck.runner.ConsoleRunner, which takes a session-parameters file and group-list file.
• Added runtime/opencmis-tck/pom.xml as the local Maven runtime descriptor.
• Documented the retired Apache Chemistry boundary, Maven Central source, and OpenCMIS session parameter keys in docs/LOCAL-TCK-RUNTIME.md.

D2.2 - Local Environment Bootstrap Command

id: OPEN-CMIS-TCK-WP-0002-T002
status: done
priority: high
state_hub_task_id: "f993c1ef-8e6f-4ad1-8375-4487887deb8b"

Acceptance:

• Provide a single local command or script that checks Java, Maven, network/cache access, and OpenCMIS TCK runtime availability.
• The bootstrap does not commit downloaded artifacts or secrets to the repo.
• Missing prerequisites produce actionable diagnostics.
• Successful bootstrap writes a local runtime summary that can be referenced by later runs.

Progress:

• Added scripts/bootstrap_opencmis_tck.py and open_cmis_tck.bootstrap.check_runtime.
• The bootstrap writes .local/opencmis-tck/runtime-summary.json and can optionally resolve Maven dependencies with --resolve.
• Current WSL posture is blocked because java and mvn are not on PATH.
• Added docs/LOCAL-RUNBOOK.md with the local sequence from dry-run adapter check through Java/Maven install, Maven dependency resolution, and first real guide-board run.
• Added regression coverage for both the blocked local posture and a ready path using a fake Java/Maven toolchain, so the command behavior is stable while the actual workstation prerequisites remain to be installed.
• Added scripts/install_local_toolchain.py as a no-sudo path that installs a local JDK/Maven toolchain under .local/toolchains, writes env.sh, and keeps downloaded runtime assets out of version control.
• Ran the local installer on this WSL workspace. It installed Temurin JDK 17.0.19 and Apache Maven 3.9.11 under .local/toolchains.
• Ran scripts/bootstrap_opencmis_tck.py --resolve with the local toolchain; the runtime summary reports ready and Maven dependency resolution succeeded.

D2.3 - OpenCMIS TCK Adapter Invocation

id: OPEN-CMIS-TCK-WP-0002-T003
status: done
priority: high
state_hub_task_id: "a446a80f-fc63-4ea8-9720-9294db57ade9"

Acceptance:

• Implement an adapter command that can invoke selected real OpenCMIS TCK checks from guide-board check_group values.
• The adapter accepts Browser Binding URL, repository ID, credentials reference or anonymous mode, timeout, output directory, and selected check group.
• Raw stdout, stderr, TCK logs, machine-readable results, and invocation metadata are captured under the run directory.
• The adapter can run a minimal repository/type group against a live target.

Progress:

• Added adapters/opencmis_console_adapter.py.
• The adapter writes OpenCMIS Browser Binding session properties, group lists, invocation metadata, and ConsoleRunner stdout/stderr under the run artifact directory.
• The baseline assessment profile now points the OpenCMIS wrapper at this local adapter command.
• Added profiles/assessments/cmis-browser-local-dry-run.json, which exercises guide-board, preflight, wrapper, ConsoleRunner adapter, and assessment-package artifact capture without requiring Java/Maven.
• The guide-board dry-run captures redacted session properties and group lists as fingerprinted assessment artifacts.
• Local Java/Maven are now installed under .local/toolchains, and Maven has resolved the OpenCMIS TCK runtime.
• Ran a non-dry ConsoleRunner adapter smoke against an intentionally unreachable local URL. Maven launched the real TCK runner, captured session properties, group list, invocation metadata, stdout, and stderr, and correctly returned infrastructure_error because no CMIS target was reachable.
• Live repository/type execution remains open until a reachable CMIS Browser Binding target is available.
• Added scripts/opencmis_inmemory_server.py, which prepares and launches the Apache Chemistry OpenCMIS in-memory server WAR under Tomcat 9 as a controlled local CMIS Browser Binding pilot target.
• The launcher installs Java EE API compatibility jars into the local Tomcat runtime because OpenCMIS 1.1.0 expects Java 8-era JAX-WS classes that are not present in JDK 17.
• Added profiles/targets/opencmis-inmemory-local.json and profiles/assessments/cmis-browser-inmemory-pilot.json for a local repository/type TCK pilot against repository A1.
• Added credential-aware preflight support so authenticated CMIS targets can be probed before the TCK adapter runs.
• Ran the repository/type group through guide-board against the local OpenCMIS in-memory Browser Binding target. The real ConsoleRunner adapter completed with return code 0, captured raw stdout/stderr and metadata, and normalized 51 TCK case results.

D2.4 - Target Profiles And Credential References

id: OPEN-CMIS-TCK-WP-0002-T004
status: done
priority: high
state_hub_task_id: "c33a4d9a-a398-4a02-93c2-1d38e62a578f"

Acceptance:

• Provide local target profile templates for anonymous, basic-auth, and externally referenced credential modes without storing secrets.
• Add clear examples for a kontextual-engine compatibility endpoint and a generic CMIS Browser Binding endpoint.
• Profile diagnostics confirm endpoint URL, binding, repository ID, expected capability groups, known gaps, and runtime timeout posture.
• Documentation explains how to run a profile validation before starting a TCK run.

Progress:

• Added anonymous, environment-credential, and file-credential target profile templates under profiles/targets/templates/.
• Added credential reference diagnostics to open_cmis_tck.profile.
• Wired credentials_ref and target profile directory expansion into the OpenCMIS wrapper command placeholders.
• Updated the ConsoleRunner adapter to support env: and file: credential refs without retaining raw passwords in evidence artifacts.

D2.5 - Real Result Normalization

id: OPEN-CMIS-TCK-WP-0002-T005
status: done
priority: high
state_hub_task_id: "03ba9506-cce8-44fa-a036-2ab1ffc7a176"

Acceptance:

• Normalize actual OpenCMIS TCK output into guide-board evidence for each selected check group.
• Preserve real TCK case IDs, class names, failure messages, skipped tests, and raw artifact paths.
• Distinguish pass, fail, expected gap, unsupported by design, blocked, and infrastructure error from real output.
• Add fixtures derived from real sanitized output so normalization remains regression-tested.

Progress:

• Added open_cmis_tck.normalization as the shared OpenCMIS result parser.
• The ConsoleRunner adapter now parses Apache Chemistry TextReport stdout and emits case-level evidence with native status, guide-board status, group/test names, source location, selected check group, and group class facts.
• The guide-board wrapper now accepts native TextReport output directly in addition to adapter JSON and JUnit-style XML.
• Added a sanitized TextReport-format fixture and regression tests for direct parsing and guide-board run normalization.
• Documented the TextReport normalization path in the runner and local runtime docs.

D2.6 - Live Pilot Run

id: OPEN-CMIS-TCK-WP-0002-T006
status: done
priority: high
state_hub_task_id: "d9eb9384-3352-4b71-9918-57282ee00411"

Acceptance:

• Run the local infrastructure against at least one reachable CMIS Browser Binding target.
• Produce a guide-board run directory with evidence, findings, mappings, assessment package, retention summary, and Markdown report.
• The report clearly states which CMIS capability groups passed, failed, were blocked by infrastructure, or were unsupported by design.
• Known gaps do not hide unexpected failures in the same capability group.

Progress:

• Local Java/Maven/TCK runtime is ready via .local/toolchains.
• Probed the configured kontextual-cmis-compat Browser Binding URL http://127.0.0.1:8000/cmis/compat-tck/browser; it currently returns HTTP 404, so the CMIS target is not reachable at the profile URL.
• Ran a guide-board baseline attempt into .local/runs/open-cmis-tck-live-attempt; it produced a run directory, assessment package, Markdown report, retention summary, and CMIS maturity scorecard with status infrastructure_error at preflight.
• Live pilot execution is blocked on starting or providing a reachable CMIS Browser Binding endpoint and, if needed, credentials.
• Added a controlled OpenCMIS in-memory pilot target path so the first live TCK run can proceed locally even before the actual system endpoint is available.
• Started the local OpenCMIS in-memory server under Tomcat 9 with Java EE API compatibility jars for JDK 17.
• Ran cmis-browser-inmemory-pilot against http://127.0.0.1:18080/inmemory/browser using environment-backed sample credentials.
• The guide-board run .local/runs/opencmis-inmemory-pilot completed and produced normalized evidence, mappings, findings, assessment package, Markdown report, retention summary, and CMIS maturity scorecard.
• Pilot result: preflight passed; repository/type TCK completed with 36 pass, 13 info, and 2 warning case results. Scorecard marks repository/type as partial pending warning review.

D2.7 - CMIS Capability Maturity Scorecard

id: OPEN-CMIS-TCK-WP-0002-T007
status: done
priority: high
state_hub_task_id: "7365052f-0d76-4fb2-b32c-236476b0f937"

Acceptance:

• Define a small maturity model for CMIS Browser Binding capability groups.
• Score repository/type, object/content, navigation, query, relationships, ACL/policy, versioning, change log, and extension/known-gap posture.
• Separate conformance evidence from maturity interpretation.
• Generate a compact JSON scorecard and Markdown summary suitable for downstream product-quality tracking.

Progress:

• Added open_cmis_tck.scorecard and scripts/cmis_scorecard.py.
• The scorecard reads guide-board evidence, findings, mappings, and assessment package outputs and writes JSON/Markdown reports.
• Added docs/CMIS-MATURITY-SCORECARD.md and linked scorecard generation from the local runbook.

D2.8 - Operator Documentation

id: OPEN-CMIS-TCK-WP-0002-T008
status: done
priority: medium
state_hub_task_id: "da7022de-3d05-43bf-9c06-91f00c159bdc"

Acceptance:

• Document the local setup flow from clean checkout to first live TCK run.
• Include troubleshooting for Java/Maven, TCK artifact resolution, endpoint reachability, repository ID mismatch, credentials, timeouts, and failed tests.
• Provide copy-pasteable commands for local execution through guide-board.
• Clearly mark the formal-certification boundary.

Progress:

• Added docs/LOCAL-RUNBOOK.md with dry-run, Java/Maven install, TCK runtime resolution, real guide-board run, authenticated target, and troubleshooting commands.
• Linked the runbook from README and local runtime documentation.

D2.9 - Container/Test Host Handoff Notes

id: OPEN-CMIS-TCK-WP-0002-T009
status: done
priority: low
state_hub_task_id: "254d099a-5406-43b0-9497-9af594a9b911"

Acceptance:

• Capture what would need to move into a container or dedicated test host after local execution is stable.
• Identify mounts for profiles, credentials, cached TCK assets, and run output.
• Identify environment variables and network assumptions.
• Do not implement the container/test-host move in this workplan unless local execution is already proven.

Progress:

• Added docs/CONTAINER-HANDOFF.md with runtime-image contents, mounts, environment variables, network assumptions, and local-readiness handoff criteria.

Definition Of Done

• A developer can run one documented local bootstrap command.
• A developer can point a profile at a CMIS Browser Binding endpoint and run at least one real OpenCMIS TCK check group through guide-board.
• The run produces normalized evidence, raw artifacts, findings, mappings, retention summary, assessment package, and Markdown report.
• The output explains what is working, what is failing, what is blocked by infrastructure, and what is unsupported by design.
• A first CMIS capability maturity scorecard is generated from evidence.
• Container/test-host migration is documented as a follow-on path, not a hidden prerequisite for local use.