From 6382a5a7ab67aaefada4c28cc68a2a766309bf8d Mon Sep 17 00:00:00 2001 From: tegwick Date: Fri, 8 May 2026 10:26:18 +0200 Subject: [PATCH] First guide-board CMIS assessment run --- docs/cmis-1-1-capability-scorecard.md | 7 + ...ncmis-tck-assessment-2026-05-08T063312Z.md | 277 ++++++++++++++++++ ...-cmis-browser-binding-tck-compatibility.md | 249 ++++++++++++++++ 3 files changed, 533 insertions(+) create mode 100644 docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md create mode 100644 workplans/KONT-WP-0013-cmis-browser-binding-tck-compatibility.md diff --git a/docs/cmis-1-1-capability-scorecard.md b/docs/cmis-1-1-capability-scorecard.md index c2a0653..3729732 100644 --- a/docs/cmis-1-1-capability-scorecard.md +++ b/docs/cmis-1-1-capability-scorecard.md @@ -2,6 +2,13 @@ Date: 2026-05-07 +Evidence update: the 2026-05-08 OpenCMIS TCK run found that broad commodity +CMIS client compatibility is currently pre-compliance because OpenCMIS cannot +create a Browser Binding session. See +`docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md`. The score below is +therefore retained as the pre-TCK product-depth estimate, not as external CMIS +compatibility evidence. + Status: baseline scorecard for the current Browser Binding subset. ## Purpose diff --git a/docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md b/docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md new file mode 100644 index 0000000..6ed3666 --- /dev/null +++ b/docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md @@ -0,0 +1,277 @@ +# CMIS OpenCMIS TCK Assessment + +Run timestamp: 2026-05-08T06:33:12Z +Local timestamp: 2026-05-08T08:33:12+02:00 +Status: evidence-backed compatibility blocker found + +## Purpose + +Persist the first real `guide-board` + `open-cmis-tck` assessment against a +running `kontextual-engine` CMIS access point. This document supersedes prior +pre-TCK estimates for broad commodity CMIS client compatibility. + +The assessment distinguishes two separate facts: + +- The native Kontextual CMIS-inspired route layer exposes useful document, + content-stream, query, ACL, relationship, parent, and profile behavior. +- The current endpoint is not yet consumable as a standard CMIS 1.1 Browser + Binding endpoint by Apache Chemistry OpenCMIS. + +## Scope + +Target repository: `kontextual-engine` + +External test facility: + +- `guide-board` +- `open-cmis-tck` +- Apache Chemistry OpenCMIS TCK 1.1.0 +- Browser Binding profile: `cmis-browser-baseline` + +Runtime target used for this run: + +- URL: `http://127.0.0.1:8010/cmis/compat-tck/browser` +- Access point: `compat-tck` +- Reported repository ID: `kontextual-compat-tck` +- TCK profile repository ID: `compat-tck` + +The production profile file currently points to port `8000`; this run used a +temporary copy redirected to port `8010` because port `8000` was already owned +by another local service that returned 404 for the CMIS probes. + +## Commands Run + +Validate the `open-cmis-tck` guide-board extension: + +```bash +cd /home/worsch/guide-board +PYTHONPATH=src python3 -m guide_board --extension-dir ../open-cmis-tck extensions validate +``` + +Run the live OpenCMIS baseline against the temporary target profile: + +```bash +cd /home/worsch/guide-board +source /home/worsch/open-cmis-tck/.local/toolchains/env.sh +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + run \ + --target /tmp/kontextual-cmis-compat-8010.json \ + --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \ + --output-dir /tmp/open-cmis-tck-kontextual-live-20260508 +``` + +Generate the scorecard: + +```bash +cd /home/worsch/open-cmis-tck +PYTHONPATH=src python3 scripts/cmis_scorecard.py \ + --run-dir /tmp/open-cmis-tck-kontextual-live-20260508 +``` + +Run focused internal CMIS tests with service dependencies available: + +```bash +cd /home/worsch/kontextual-engine +.venv/bin/python -m pytest tests/cmis --perf-history-disable +``` + +## External TCK Result + +Guide Board live run: + +- Run ID: `run-20260508T063312Z` +- Assessment status: `infrastructure_error` +- Summary: `pass: 1`, `infrastructure_error: 2` +- Preflight: pass; endpoint was reachable and returned parseable JSON. +- Repository/type TCK group: blocked before test cases could be parsed. +- Object/content TCK group: blocked before test cases could be parsed. + +Generated scorecard: + +- Maturity score: `9.52` +- Coverage: `2/9` groups, `22.22%` +- Repository and type metadata: blocked, score `1/4` +- Object and content services: blocked, score `1/4` +- Navigation, query, relationships, ACL/policy, versioning, change log, and + extensions: not assessed. + +Primary blocker from OpenCMIS ConsoleRunner: + +```text +Found invalid Repository Info! +``` + +The same failure occurs in: + +- `BasicsTestGroup` and `TypesTestGroup` +- `CRUDTestGroup` + +No OpenCMIS semantic cases were parsed because the OpenCMIS client could not +establish a valid CMIS Browser Binding session. + +## Evidence Artifacts + +Run artifacts were generated under `/tmp` and may be ephemeral. The critical +facts are copied into this document. + +- `/tmp/open-cmis-tck-kontextual-live-20260508/reports/report.md` +- `/tmp/open-cmis-tck-kontextual-live-20260508/reports/cmis-maturity-scorecard.md` +- `/tmp/open-cmis-tck-kontextual-live-20260508/normalized/evidence.json` +- `/tmp/open-cmis-tck-kontextual-live-20260508/normalized/findings.json` +- `/tmp/open-cmis-tck-kontextual-live-20260508/artifacts/open-cmis-tck/preflight/response-body.bin` +- `/tmp/open-cmis-tck-kontextual-live-20260508/artifacts/open-cmis-tck/tck/repository-type/console-runner-stdout.txt` +- `/tmp/open-cmis-tck-kontextual-live-20260508/artifacts/open-cmis-tck/tck/object-content/console-runner-stdout.txt` + +## Repository Info Shape Observed + +Preflight observed a JSON object with these top-level keys: + +```text +binding +capabilities +cmis_version_supported +compliance +principal_anonymous +principal_anyone +product_name +product_version +profile +repository_description +repository_features +repository_id +repository_name +root_folder_id +unsupported_features +vendor_name +``` + +The endpoint reports: + +- `repository_id`: `kontextual-compat-tck` +- `repository_name`: `Kontextual Engine compat-tck` +- `cmis_version_supported`: `1.1` +- `root_folder_id`: `cmis-root` +- `binding`: `browser` + +The capability map also uses snake_case keys, for example: + +- `capability_query` +- `capability_acl` +- `capability_content_stream_updatability` +- `capability_multifiling` + +OpenCMIS expects CMIS Browser Binding repository information/service document +shape, including CMIS field names such as: + +- `repositoryId` +- `repositoryName` +- `cmisVersionSupported` +- `rootFolderId` +- Browser Binding capability names in the shape expected by the OpenCMIS client + +The assessment therefore classifies the immediate blocker as protocol contract +shape, not absence of all underlying engine capabilities. + +## Internal CMIS Test Result + +Focused internal test run: + +- Command: `.venv/bin/python -m pytest tests/cmis --perf-history-disable` +- Result: `42 passed`, `3 failed` + +Passing areas include: + +- access profile modeling +- compliance flags +- contract examples +- domain mapper +- fixture integration +- runtime Browser Binding behavior + +Failures: + +1. OpenAPI generation fails around the streaming route return annotation. + Pydantic reports that the `StreamingResponse` forward reference is not fully + defined when generating `/openapi.json`. +2. `test_cmis_readonly_children_object_content_query_relationships_and_changes` + expects root `/children` to include documents directly. Current behavior + returns first-level projected folders at root. +3. `test_cmis_profile_gates_visibility_by_access_point` has the same root + `/children` expectation for admin/export visibility. + +The navigation failures are contract alignment issues: either root children +should expose documents directly for the CMIS profile, or the tests should ask +for the folder that contains those projected documents. The OpenAPI failure is +a concrete implementation bug. + +## Native Route Probe Result + +Direct live probes confirmed useful native behavior: + +- `/cmis` lists multiple profiled access points. +- `/cmis/compat-tck/browser` returns repository metadata and conservative + unsupported-feature diagnostics. +- `/cmis/compat-tck/browser/types` returns six type projections. +- `/cmis/compat-tck/browser/children` returns synthetic root folders. +- `/cmis/compat-tck/browser/query?q=SELECT * FROM cmis:document` returns + document projections. +- `/cmis/compat-tck/browser/object/{object_id}` returns object properties, + content stream metadata, allowable actions, version projection, and ACL + projection. +- `/cmis/compat-tck/browser/content-bytes/{object_id}` returns real + `text/markdown` bytes. +- `/cmis/compat-tck/browser/parents/{object_id}` returns projection-only parent + folders. +- Unsupported query/property-update cases return structured diagnostics. + +This confirms that content-stream and object-projection foundations exist. The +OpenCMIS blocker is the external CMIS Browser Binding adapter contract. + +## Capability Interpretation + +Evidence-backed commodity CMIS compatibility is currently lower than the +pre-TCK estimate: + +- Broad OpenCMIS client compatibility: `pre-compliance`, approximately + `0-10%` until session creation succeeds. +- Harness maturity score from the live run: `9.52`. +- Controlled-client usefulness of the native CMIS-inspired routes remains + materially higher, approximately `55-60%`, because known clients can use the + documented JSON routes and profile semantics. + +The previous `docs/cmis-1-1-capability-scorecard.md` estimate remains useful as +a product-depth hypothesis, but it must not be presented as external CMIS +compatibility evidence until OpenCMIS can connect and execute test cases. + +## Findings + +| ID | Severity | Finding | Evidence | Required response | +| --- | --- | --- | --- | --- | +| CMIS-TCK-20260508-001 | Critical | OpenCMIS cannot create a Browser Binding session because repository info is invalid for CMIS expectations. | ConsoleRunner: `Found invalid Repository Info!` | Implement CMIS-shaped service document/repository info and rerun repository/type group. | +| CMIS-TCK-20260508-002 | High | The endpoint and TCK target disagree on repository ID. | Endpoint reports `kontextual-compat-tck`; TCK profile uses `compat-tck`. | Align repository ID or provide a stable alias before rerunning TCK. | +| CMIS-TCK-20260508-003 | High | Browser Binding response fields use snake_case/native DTO shape instead of CMIS Browser Binding shape. | Preflight top-level keys and capability keys are snake_case. | Add CMIS protocol serializers that translate from native DTOs to CMIS Browser Binding JSON. | +| CMIS-TCK-20260508-004 | Medium | `/openapi.json` generation fails for the content byte streaming route. | Focused internal CMIS suite failure in `test_cmis_browser_binding_routes_are_advertised_in_openapi`. | Fix FastAPI route metadata for `StreamingResponse`. | +| CMIS-TCK-20260508-005 | Medium | Root `/children` contract is ambiguous between synthetic-folder discovery and direct document listing. | Focused internal CMIS suite failures for readonly and admin/export children expectations. | Decide CMIS root behavior and align implementation/tests with Browser Binding expectations. | +| CMIS-TCK-20260508-006 | Medium | The TCK facility reports target protocol failures as generic `infrastructure_error`. | Normalized findings classify both TCK groups as infrastructure errors. | Improve classifier/preflight in `open-cmis-tck` so invalid repository info is reported as target protocol shape failure. | + +## Recommended Fix Sequence + +1. Define exact CMIS Browser Binding service-root/repository-info examples based + on OpenCMIS expectations. +2. Implement a separate CMIS protocol serialization layer rather than exposing + native snake_case DTOs directly. +3. Align `compat-tck` repository ID and target profile. +4. Make preflight reject native-shaped repository info before invoking Maven. +5. Fix OpenAPI generation for streaming routes. +6. Resolve root navigation semantics and add explicit tests for root, + synthetic folders, and object-by-path behavior. +7. Rerun only the repository/type group until OpenCMIS session creation passes. +8. Then enable object/content TCK and work through semantic failures. +9. Update the public scorecard only from TCK-backed evidence. + +## Boundary + +This assessment does not claim CMIS certification or full conformance. It is a +local engineering assessment using Apache Chemistry OpenCMIS TCK as the first +external compatibility oracle. diff --git a/workplans/KONT-WP-0013-cmis-browser-binding-tck-compatibility.md b/workplans/KONT-WP-0013-cmis-browser-binding-tck-compatibility.md new file mode 100644 index 0000000..8438082 --- /dev/null +++ b/workplans/KONT-WP-0013-cmis-browser-binding-tck-compatibility.md @@ -0,0 +1,249 @@ +--- +id: KONT-WP-0013 +type: workplan +title: "CMIS Browser Binding TCK Compatibility" +domain: markitect +repo: kontextual-engine +status: planned +owner: codex +topic_slug: markitect +planning_priority: high +planning_order: 13 +created: "2026-05-08" +updated: "2026-05-08" +state_hub_workstream_id: "18710625-e573-45b1-9b0a-a8b43caaada1" +--- + +# KONT-WP-0013: CMIS Browser Binding TCK Compatibility + +## Purpose + +Turn the current CMIS-inspired adapter into a CMIS 1.1 Browser Binding surface +that Apache Chemistry OpenCMIS can connect to and assess. The immediate goal is +not full CMIS certification; it is to pass session creation and then drive +repository/type plus object/content behavior from real TCK evidence. + +## Trigger + +The 2026-05-08 live `guide-board` + `open-cmis-tck` run reached the endpoint, +but OpenCMIS failed before semantic test cases could execute: + +```text +Found invalid Repository Info! +``` + +The result is persisted in +`docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md`. + +## References + +- `docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md` +- `docs/cmis-1-1-capability-scorecard.md` +- `docs/cmis-compliance-assessment.md` +- `docs/cmis-profiled-access-points-implementation.md` +- `src/kontextual_engine/api/app.py` +- `src/kontextual_engine/core/cmis.py` +- `tests/cmis/test_cmis_browser_binding_api.py` +- `tests/cmis/test_cmis_runtime_browser_binding.py` +- `/home/worsch/open-cmis-tck/profiles/targets/kontextual-cmis-compat.json` +- `/home/worsch/open-cmis-tck/profiles/assessments/cmis-browser-baseline.json` + +## Boundary + +This workplan fixes Browser Binding compatibility for the profiled CMIS +connector. It must not turn CMIS into a second domain model. Native engine DTOs, +policy, blob handling, audit, and repository services remain authoritative; the +CMIS layer translates those capabilities into standard protocol shapes. + +AtomPub and Web Services remain unsupported by design. Mutating multifiling, +PWC/versioning services, broad CMIS SQL, ACL mutation, policies, renditions, +retention/hold, and bulk updates remain out of scope unless a later workplan +adds them explicitly. + +## Desired Outcome + +- OpenCMIS Browser Binding session creation succeeds against `compat-tck`. +- Repository/type TCK group executes and produces parsed cases instead of an + infrastructure/protocol blocker. +- Object/content TCK group executes far enough to expose true semantic gaps. +- Internal CMIS route tests pass with service dependencies installed. +- Updated CMIS scorecard separates TCK-backed compatibility from native + controlled-client usefulness. + +## F13.1 - Define Browser Binding protocol contract examples + +```task +id: KONT-WP-0013-T001 +status: todo +priority: high +state_hub_task_id: "fa2f99fb-51ed-4fba-8f36-ba4b790908e5" +``` + +Acceptance: + +- Repository service document/repository-info examples are documented in the + exact shape expected by OpenCMIS Browser Binding. +- Required field names, capability names, repository ID handling, root folder + ID handling, and service URLs are covered. +- The examples include at least one valid `compat-tck` repository response and + one invalid native-shaped response used as a negative preflight fixture. + +## F13.2 - Implement CMIS Browser Binding serializers + +```task +id: KONT-WP-0013-T002 +status: todo +priority: high +state_hub_task_id: "39949f5b-32bb-4ec7-b0f6-83a16ce443b7" +``` + +Acceptance: + +- Repository info, capabilities, type definitions, object envelopes, property + maps, content stream descriptors, ACL projections, and change entries are + emitted through CMIS protocol serializers. +- Native snake_case DTOs are no longer exposed directly on Browser Binding + compatibility routes. +- Serializer tests assert CMIS field names such as `repositoryId`, + `repositoryName`, `cmisVersionSupported`, and `rootFolderId`. + +## F13.3 - Align target repository identity and access-point profile + +```task +id: KONT-WP-0013-T003 +status: todo +priority: high +state_hub_task_id: "96e11fe6-68c6-46ce-88b8-bda0b9baf2d5" +``` + +Acceptance: + +- The `compat-tck` access point and `open-cmis-tck` target profile agree on the + repository ID. +- If an alias is needed, it is explicit, deterministic, and documented. +- The port/profile assumptions for local TCK runs are documented without + relying on accidental services already bound to port `8000`. + +## F13.4 - Add protocol-shape preflight gates + +```task +id: KONT-WP-0013-T004 +status: todo +priority: high +state_hub_task_id: "06506455-b3c3-4f17-ae8a-8ffebc226f73" +``` + +Acceptance: + +- `open-cmis-tck` preflight fails native-shaped repository info before invoking + Maven. +- The failure is classified as target protocol shape incompatibility, not a + generic infrastructure error. +- The preflight reports actionable missing or misshaped fields. + +## F13.5 - Fix OpenAPI generation for streaming routes + +```task +id: KONT-WP-0013-T005 +status: todo +priority: medium +state_hub_task_id: "de73a3ff-ddf9-40bb-b89c-50d6fabdb841" +``` + +Acceptance: + +- `/openapi.json` renders successfully with CMIS byte-stream routes enabled. +- The route still returns a real streaming response at runtime. +- `tests/cmis/test_cmis_browser_binding_api.py::test_cmis_browser_binding_routes_are_advertised_in_openapi` + passes with service dependencies installed. + +## F13.6 - Resolve navigation contract and root children behavior + +```task +id: KONT-WP-0013-T006 +status: todo +priority: medium +state_hub_task_id: "1cf8d36c-6127-46cd-b409-dda7a045b8dd" +``` + +Acceptance: + +- Root folder behavior is explicitly defined for Browser Binding compatibility. +- Tests cover root children, synthetic folder children, projected parents, and + hidden object filtering. +- Internal route tests and OpenCMIS expectations do not disagree about whether + root returns only folders or direct documents. + +## F13.7 - Establish OpenCMIS session smoke test + +```task +id: KONT-WP-0013-T007 +status: todo +priority: high +state_hub_task_id: "38777aea-7d2d-4b30-99b4-5194f2c2f8b0" +``` + +Acceptance: + +- A fast smoke check proves that OpenCMIS can list repositories and retrieve + repository info before full TCK groups run. +- The smoke check is captured by `guide-board`/`open-cmis-tck` artifacts. +- Failure messages preserve the underlying OpenCMIS exception. + +## F13.8 - Rerun repository/type TCK group to parsed-case evidence + +```task +id: KONT-WP-0013-T008 +status: todo +priority: high +state_hub_task_id: "8e7b2235-d39a-465a-b828-b05eb6bcfce3" +``` + +Acceptance: + +- `repository-type` no longer exits at session creation. +- Basics and Types test cases are parsed and mapped into normalized evidence. +- Any semantic failures become concrete follow-up tasks with capability labels. + +## F13.9 - Rerun object/content TCK group and classify gaps + +```task +id: KONT-WP-0013-T009 +status: todo +priority: high +state_hub_task_id: "22af3e39-c217-4f6b-b54a-5ecea3894020" +``` + +Acceptance: + +- `object-content` reaches parsed TCK cases. +- Failures are separated into supported-fix, unsupported-by-design, and profile + expectation categories. +- Content stream read/write behavior is tested against both internal route + tests and OpenCMIS expectations. + +## F13.10 - Update scorecard and compatibility documentation + +```task +id: KONT-WP-0013-T010 +status: todo +priority: medium +state_hub_task_id: "469a3ea6-a6ab-4e6e-ba4f-5b1abdabd6c4" +``` + +Acceptance: + +- `docs/cmis-1-1-capability-scorecard.md` is updated from TCK-backed evidence. +- The scorecard separates broad CMIS client compatibility from controlled + native-profile usefulness. +- Deployment docs explain which CMIS clients are expected to work and which + capabilities are honestly unsupported. + +## Definition Of Done + +- Focused internal CMIS tests pass with service dependencies installed. +- OpenCMIS can establish a Browser Binding session against `compat-tck`. +- Repository/type TCK group produces parsed evidence. +- Object/content TCK group produces parsed evidence or documented + unsupported-by-design gaps. +- Updated documentation clearly states the evidence-backed CMIS posture.