# CMIS OpenCMIS TCK Assessment

Run timestamp: 2026-05-08T06:33:12Z
Local timestamp: 2026-05-08T08:33:12+02:00
Status: evidence-backed compatibility blocker found

Superseded implementation evidence: the blocker described here was resolved by
`KONT-WP-0013`. See
`docs/cmis-opencmis-tck-implementation-evidence-2026-05-08T092113Z.md` for the
latest completed OpenCMIS run.

## Purpose

Persist the first real `guide-board` + `open-cmis-tck` assessment against a
running `kontextual-engine` CMIS access point. This document supersedes prior
pre-TCK estimates for broad commodity CMIS client compatibility.

The assessment distinguishes two separate facts:

- The native Kontextual CMIS-inspired route layer exposes useful document,
  content-stream, query, ACL, relationship, parent, and profile behavior.
- The current endpoint is not yet consumable as a standard CMIS 1.1 Browser
  Binding endpoint by Apache Chemistry OpenCMIS.

## Scope

Target repository: `kontextual-engine`

External test facility:

- `guide-board`
- `open-cmis-tck`
- Apache Chemistry OpenCMIS TCK 1.1.0
- Browser Binding profile: `cmis-browser-baseline`

Runtime target used for this run:

- URL: `http://127.0.0.1:8010/cmis/compat-tck/browser`
- Access point: `compat-tck`
- Reported repository ID: `kontextual-compat-tck`
- TCK profile repository ID: `compat-tck`

The production profile file currently points to port `8000`; this run used a
temporary copy redirected to port `8010` because port `8000` was already owned
by another local service that returned 404 for the CMIS probes.

## Commands Run

Validate the `open-cmis-tck` guide-board extension:

```bash
cd /home/worsch/guide-board
PYTHONPATH=src python3 -m guide_board --extension-dir ../open-cmis-tck extensions validate
```

Run the live OpenCMIS baseline against the temporary target profile:

```bash
cd /home/worsch/guide-board
source /home/worsch/open-cmis-tck/.local/toolchains/env.sh
PYTHONPATH=src python3 -m guide_board \
  --extension-dir ../open-cmis-tck \
  run \
  --target /tmp/kontextual-cmis-compat-8010.json \
  --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \
  --output-dir /tmp/open-cmis-tck-kontextual-live-20260508
```

Generate the scorecard:

```bash
cd /home/worsch/open-cmis-tck
PYTHONPATH=src python3 scripts/cmis_scorecard.py \
  --run-dir /tmp/open-cmis-tck-kontextual-live-20260508
```

Run focused internal CMIS tests with service dependencies available:

```bash
cd /home/worsch/kontextual-engine
.venv/bin/python -m pytest tests/cmis --perf-history-disable
```

## External TCK Result

Guide Board live run:

- Run ID: `run-20260508T063312Z`
- Assessment status: `infrastructure_error`
- Summary: `pass: 1`, `infrastructure_error: 2`
- Preflight: pass; endpoint was reachable and returned parseable JSON.
- Repository/type TCK group: blocked before test cases could be parsed.
- Object/content TCK group: blocked before test cases could be parsed.

Generated scorecard:

- Maturity score: `9.52`
- Coverage: `2/9` groups, `22.22%`
- Repository and type metadata: blocked, score `1/4`
- Object and content services: blocked, score `1/4`
- Navigation, query, relationships, ACL/policy, versioning, change log, and
  extensions: not assessed.

Primary blocker from OpenCMIS ConsoleRunner:

```text
Found invalid Repository Info!
```

The same failure occurs in:

- `BasicsTestGroup` and `TypesTestGroup`
- `CRUDTestGroup`

No OpenCMIS semantic cases were parsed because the OpenCMIS client could not
establish a valid CMIS Browser Binding session.

## Evidence Artifacts

Run artifacts were generated under `/tmp` and may be ephemeral. The critical
facts are copied into this document.

- `/tmp/open-cmis-tck-kontextual-live-20260508/reports/report.md`
- `/tmp/open-cmis-tck-kontextual-live-20260508/reports/cmis-maturity-scorecard.md`
- `/tmp/open-cmis-tck-kontextual-live-20260508/normalized/evidence.json`
- `/tmp/open-cmis-tck-kontextual-live-20260508/normalized/findings.json`
- `/tmp/open-cmis-tck-kontextual-live-20260508/artifacts/open-cmis-tck/preflight/response-body.bin`
- `/tmp/open-cmis-tck-kontextual-live-20260508/artifacts/open-cmis-tck/tck/repository-type/console-runner-stdout.txt`
- `/tmp/open-cmis-tck-kontextual-live-20260508/artifacts/open-cmis-tck/tck/object-content/console-runner-stdout.txt`

## Repository Info Shape Observed

Preflight observed a JSON object with these top-level keys:

```text
binding
capabilities
cmis_version_supported
compliance
principal_anonymous
principal_anyone
product_name
product_version
profile
repository_description
repository_features
repository_id
repository_name
root_folder_id
unsupported_features
vendor_name
```

The endpoint reports:

- `repository_id`: `kontextual-compat-tck`
- `repository_name`: `Kontextual Engine compat-tck`
- `cmis_version_supported`: `1.1`
- `root_folder_id`: `cmis-root`
- `binding`: `browser`

The capability map also uses snake_case keys, for example:

- `capability_query`
- `capability_acl`
- `capability_content_stream_updatability`
- `capability_multifiling`

OpenCMIS expects CMIS Browser Binding repository information/service document
shape, including CMIS field names such as:

- `repositoryId`
- `repositoryName`
- `cmisVersionSupported`
- `rootFolderId`
- Browser Binding capability names in the shape expected by the OpenCMIS client

The assessment therefore classifies the immediate blocker as protocol contract
shape, not absence of all underlying engine capabilities.

## Internal CMIS Test Result

Focused internal test run:

- Command: `.venv/bin/python -m pytest tests/cmis --perf-history-disable`
- Result: `42 passed`, `3 failed`

Passing areas include:

- access profile modeling
- compliance flags
- contract examples
- domain mapper
- fixture integration
- runtime Browser Binding behavior

Failures:

1. OpenAPI generation fails around the streaming route return annotation.
   Pydantic reports that the `StreamingResponse` forward reference is not fully
   defined when generating `/openapi.json`.
2. `test_cmis_readonly_children_object_content_query_relationships_and_changes`
   expects root `/children` to include documents directly. Current behavior
   returns first-level projected folders at root.
3. `test_cmis_profile_gates_visibility_by_access_point` has the same root
   `/children` expectation for admin/export visibility.

The navigation failures are contract alignment issues: either root children
should expose documents directly for the CMIS profile, or the tests should ask
for the folder that contains those projected documents. The OpenAPI failure is
a concrete implementation bug.

## Native Route Probe Result

Direct live probes confirmed useful native behavior:

- `/cmis` lists multiple profiled access points.
- `/cmis/compat-tck/browser` returns repository metadata and conservative
  unsupported-feature diagnostics.
- `/cmis/compat-tck/browser/types` returns six type projections.
- `/cmis/compat-tck/browser/children` returns synthetic root folders.
- `/cmis/compat-tck/browser/query?q=SELECT * FROM cmis:document` returns
  document projections.
- `/cmis/compat-tck/browser/object/{object_id}` returns object properties,
  content stream metadata, allowable actions, version projection, and ACL
  projection.
- `/cmis/compat-tck/browser/content-bytes/{object_id}` returns real
  `text/markdown` bytes.
- `/cmis/compat-tck/browser/parents/{object_id}` returns projection-only parent
  folders.
- Unsupported query/property-update cases return structured diagnostics.

This confirms that content-stream and object-projection foundations exist. The
OpenCMIS blocker is the external CMIS Browser Binding adapter contract.

## Capability Interpretation

Evidence-backed commodity CMIS compatibility is currently lower than the
pre-TCK estimate:

- Broad OpenCMIS client compatibility: `pre-compliance`, approximately
  `0-10%` until session creation succeeds.
- Harness maturity score from the live run: `9.52`.
- Controlled-client usefulness of the native CMIS-inspired routes remains
  materially higher, approximately `55-60%`, because known clients can use the
  documented JSON routes and profile semantics.

The previous `docs/cmis-1-1-capability-scorecard.md` estimate remains useful as
a product-depth hypothesis, but it must not be presented as external CMIS
compatibility evidence until OpenCMIS can connect and execute test cases.

## Findings

| ID | Severity | Finding | Evidence | Required response |
| --- | --- | --- | --- | --- |
| CMIS-TCK-20260508-001 | Critical | OpenCMIS cannot create a Browser Binding session because repository info is invalid for CMIS expectations. | ConsoleRunner: `Found invalid Repository Info!` | Implement CMIS-shaped service document/repository info and rerun repository/type group. |
| CMIS-TCK-20260508-002 | High | The endpoint and TCK target disagree on repository ID. | Endpoint reports `kontextual-compat-tck`; TCK profile uses `compat-tck`. | Align repository ID or provide a stable alias before rerunning TCK. |
| CMIS-TCK-20260508-003 | High | Browser Binding response fields use snake_case/native DTO shape instead of CMIS Browser Binding shape. | Preflight top-level keys and capability keys are snake_case. | Add CMIS protocol serializers that translate from native DTOs to CMIS Browser Binding JSON. |
| CMIS-TCK-20260508-004 | Medium | `/openapi.json` generation fails for the content byte streaming route. | Focused internal CMIS suite failure in `test_cmis_browser_binding_routes_are_advertised_in_openapi`. | Fix FastAPI route metadata for `StreamingResponse`. |
| CMIS-TCK-20260508-005 | Medium | Root `/children` contract is ambiguous between synthetic-folder discovery and direct document listing. | Focused internal CMIS suite failures for readonly and admin/export children expectations. | Decide CMIS root behavior and align implementation/tests with Browser Binding expectations. |
| CMIS-TCK-20260508-006 | Medium | The TCK facility reports target protocol failures as generic `infrastructure_error`. | Normalized findings classify both TCK groups as infrastructure errors. | Improve classifier/preflight in `open-cmis-tck` so invalid repository info is reported as target protocol shape failure. |

## Recommended Fix Sequence

1. Define exact CMIS Browser Binding service-root/repository-info examples based
   on OpenCMIS expectations.
2. Implement a separate CMIS protocol serialization layer rather than exposing
   native snake_case DTOs directly.
3. Align `compat-tck` repository ID and target profile.
4. Make preflight reject native-shaped repository info before invoking Maven.
5. Fix OpenAPI generation for streaming routes.
6. Resolve root navigation semantics and add explicit tests for root,
   synthetic folders, and object-by-path behavior.
7. Rerun only the repository/type group until OpenCMIS session creation passes.
8. Then enable object/content TCK and work through semantic failures.
9. Update the public scorecard only from TCK-backed evidence.

## Boundary

This assessment does not claim CMIS certification or full conformance. It is a
local engineering assessment using Apache Chemistry OpenCMIS TCK as the first
external compatibility oracle.