open-cmis-tck/docs/OPENCMIS-TCK-RUNNER.md

OpenCMIS TCK Runner

Status: draft Created: 2026-05-07

Purpose

The runner wrapper at runners/opencmis_tck.py is the boundary between guide-board and Apache Chemistry OpenCMIS TCK execution. It keeps Java/Maven setup, harness command lines, raw logs, and result normalization inside this extension.

Dependency Checks

By default, the wrapper checks:

• java -version
• mvn -version

If either dependency is unavailable, the runner returns blocked evidence with blocked_reason: missing_dependency.

Set runtime_policy.opencmis_tck.requires_java_maven to false only for tests or custom harness commands that do not use the local Java/Maven toolchain.

Command Configuration

The baseline assessment profile is configured to call the local ConsoleRunner adapter:

adapters/opencmis_console_adapter.py

See docs/LOCAL-TCK-RUNTIME.md for the bootstrap flow and Maven runtime descriptor.

You can also configure a custom TCK command as an argv list:

{
  "runtime_policy": {
    "opencmis_tck": {
      "repository_id": "compat-tck",
      "command": [
        "java",
        "-jar",
        "/assets/opencmis-tck-runner.jar",
        "--url",
        "{browser_url}",
        "--repository",
        "{repository_id}",
        "--group",
        "{check_group}",
        "--output",
        "{artifact_dir}"
      ]
    }
  }
}

Supported placeholders:

• {browser_url}
• {repository_id}
• {check_group}
• {target_id}
• {run_dir}
• {artifact_dir}

The wrapper also accepts OPENCMIS_TCK_COMMAND_JSON as a JSON string array, or OPENCMIS_TCK_COMMAND as a shell-like string that is split into argv. The final command still runs without shell expansion.

Raw Artifacts

For each selected check group, artifacts are written under:

artifacts/open-cmis-tck/tck/<check-group>/

Current artifacts:

• invocation.json
• stdout.log
• stderr.log
• normalized-runner-result.json

The guide-board core fingerprints these files in the assessment package artifact manifest when they are referenced by the runner result.

Normalization

The wrapper normalizes, in order:

1. JSON written to stdout with a tests, cases, or results array.
2. JUnit-style XML files written directly into {artifact_dir}.
3. Native OpenCMIS TextReport output written by ConsoleRunner.
4. Exit code only, when no structured output is found.

Case statuses normalize to guide-board result vocabulary: pass, fail, skipped, expected_gap, unsupported_by_design, infrastructure_error, and related core statuses.

The native text normalizer follows Apache Chemistry's TextReport structure: group headings, test headings, and result lines whose statuses are INFO, SKIPPED, OK, WARNING, FAILURE, or UNEXPECTED_EXCEPTION. It preserves the OpenCMIS status, test name, group name, source file/line where present, message, duration, selected check group, and raw artifact references.

This is enough to run the real local ConsoleRunner adapter while retaining the raw logs needed for later pilot-review refinements.