First guide-board CMIS assessment run

2026-05-08 10:26:18 +02:00
parent ec60a67df6
commit 6382a5a7ab
3 changed files with 533 additions and 0 deletions
--- a/docs/cmis-1-1-capability-scorecard.md
+++ b/docs/cmis-1-1-capability-scorecard.md
@@ -2,6 +2,13 @@

 Date: 2026-05-07

+Evidence update: the 2026-05-08 OpenCMIS TCK run found that broad commodity
+CMIS client compatibility is currently pre-compliance because OpenCMIS cannot
+create a Browser Binding session. See
+`docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md`. The score below is
+therefore retained as the pre-TCK product-depth estimate, not as external CMIS
+compatibility evidence.
+
 Status: baseline scorecard for the current Browser Binding subset.

 ## Purpose
--- a/docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md
+++ b/docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md
@@ -0,0 +1,277 @@
+# CMIS OpenCMIS TCK Assessment
+
+Run timestamp: 2026-05-08T06:33:12Z
+Local timestamp: 2026-05-08T08:33:12+02:00
+Status: evidence-backed compatibility blocker found
+
+## Purpose
+
+Persist the first real `guide-board` + `open-cmis-tck` assessment against a
+running `kontextual-engine` CMIS access point. This document supersedes prior
+pre-TCK estimates for broad commodity CMIS client compatibility.
+
+The assessment distinguishes two separate facts:
+
+- The native Kontextual CMIS-inspired route layer exposes useful document,
+  content-stream, query, ACL, relationship, parent, and profile behavior.
+- The current endpoint is not yet consumable as a standard CMIS 1.1 Browser
+  Binding endpoint by Apache Chemistry OpenCMIS.
+
+## Scope
+
+Target repository: `kontextual-engine`
+
+External test facility:
+
+- `guide-board`
+- `open-cmis-tck`
+- Apache Chemistry OpenCMIS TCK 1.1.0
+- Browser Binding profile: `cmis-browser-baseline`
+
+Runtime target used for this run:
+
+- URL: `http://127.0.0.1:8010/cmis/compat-tck/browser`
+- Access point: `compat-tck`
+- Reported repository ID: `kontextual-compat-tck`
+- TCK profile repository ID: `compat-tck`
+
+The production profile file currently points to port `8000`; this run used a
+temporary copy redirected to port `8010` because port `8000` was already owned
+by another local service that returned 404 for the CMIS probes.
+
+## Commands Run
+
+Validate the `open-cmis-tck` guide-board extension:
+
+```bash
+cd /home/worsch/guide-board
+PYTHONPATH=src python3 -m guide_board --extension-dir ../open-cmis-tck extensions validate
+```
+
+Run the live OpenCMIS baseline against the temporary target profile:
+
+```bash
+cd /home/worsch/guide-board
+source /home/worsch/open-cmis-tck/.local/toolchains/env.sh
+PYTHONPATH=src python3 -m guide_board \
+  --extension-dir ../open-cmis-tck \
+  run \
+  --target /tmp/kontextual-cmis-compat-8010.json \
+  --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \
+  --output-dir /tmp/open-cmis-tck-kontextual-live-20260508
+```
+
+Generate the scorecard:
+
+```bash
+cd /home/worsch/open-cmis-tck
+PYTHONPATH=src python3 scripts/cmis_scorecard.py \
+  --run-dir /tmp/open-cmis-tck-kontextual-live-20260508
+```
+
+Run focused internal CMIS tests with service dependencies available:
+
+```bash
+cd /home/worsch/kontextual-engine
+.venv/bin/python -m pytest tests/cmis --perf-history-disable
+```
+
+## External TCK Result
+
+Guide Board live run:
+
+- Run ID: `run-20260508T063312Z`
+- Assessment status: `infrastructure_error`
+- Summary: `pass: 1`, `infrastructure_error: 2`
+- Preflight: pass; endpoint was reachable and returned parseable JSON.
+- Repository/type TCK group: blocked before test cases could be parsed.
+- Object/content TCK group: blocked before test cases could be parsed.
+
+Generated scorecard:
+
+- Maturity score: `9.52`
+- Coverage: `2/9` groups, `22.22%`
+- Repository and type metadata: blocked, score `1/4`
+- Object and content services: blocked, score `1/4`
+- Navigation, query, relationships, ACL/policy, versioning, change log, and
+  extensions: not assessed.
+
+Primary blocker from OpenCMIS ConsoleRunner:
+
+```text
+Found invalid Repository Info!
+```
+
+The same failure occurs in:
+
+- `BasicsTestGroup` and `TypesTestGroup`
+- `CRUDTestGroup`
+
+No OpenCMIS semantic cases were parsed because the OpenCMIS client could not
+establish a valid CMIS Browser Binding session.
+
+## Evidence Artifacts
+
+Run artifacts were generated under `/tmp` and may be ephemeral. The critical
+facts are copied into this document.
+
+- `/tmp/open-cmis-tck-kontextual-live-20260508/reports/report.md`
+- `/tmp/open-cmis-tck-kontextual-live-20260508/reports/cmis-maturity-scorecard.md`
+- `/tmp/open-cmis-tck-kontextual-live-20260508/normalized/evidence.json`
+- `/tmp/open-cmis-tck-kontextual-live-20260508/normalized/findings.json`
+- `/tmp/open-cmis-tck-kontextual-live-20260508/artifacts/open-cmis-tck/preflight/response-body.bin`
+- `/tmp/open-cmis-tck-kontextual-live-20260508/artifacts/open-cmis-tck/tck/repository-type/console-runner-stdout.txt`
+- `/tmp/open-cmis-tck-kontextual-live-20260508/artifacts/open-cmis-tck/tck/object-content/console-runner-stdout.txt`
+
+## Repository Info Shape Observed
+
+Preflight observed a JSON object with these top-level keys:
+
+```text
+binding
+capabilities
+cmis_version_supported
+compliance
+principal_anonymous
+principal_anyone
+product_name
+product_version
+profile
+repository_description
+repository_features
+repository_id
+repository_name
+root_folder_id
+unsupported_features
+vendor_name
+```
+
+The endpoint reports:
+
+- `repository_id`: `kontextual-compat-tck`
+- `repository_name`: `Kontextual Engine compat-tck`
+- `cmis_version_supported`: `1.1`
+- `root_folder_id`: `cmis-root`
+- `binding`: `browser`
+
+The capability map also uses snake_case keys, for example:
+
+- `capability_query`
+- `capability_acl`
+- `capability_content_stream_updatability`
+- `capability_multifiling`
+
+OpenCMIS expects CMIS Browser Binding repository information/service document
+shape, including CMIS field names such as:
+
+- `repositoryId`
+- `repositoryName`
+- `cmisVersionSupported`
+- `rootFolderId`
+- Browser Binding capability names in the shape expected by the OpenCMIS client
+
+The assessment therefore classifies the immediate blocker as protocol contract
+shape, not absence of all underlying engine capabilities.
+
+## Internal CMIS Test Result
+
+Focused internal test run:
+
+- Command: `.venv/bin/python -m pytest tests/cmis --perf-history-disable`
+- Result: `42 passed`, `3 failed`
+
+Passing areas include:
+
+- access profile modeling
+- compliance flags
+- contract examples
+- domain mapper
+- fixture integration
+- runtime Browser Binding behavior
+
+Failures:
+
+1. OpenAPI generation fails around the streaming route return annotation.
+   Pydantic reports that the `StreamingResponse` forward reference is not fully
+   defined when generating `/openapi.json`.
+2. `test_cmis_readonly_children_object_content_query_relationships_and_changes`
+   expects root `/children` to include documents directly. Current behavior
+   returns first-level projected folders at root.
+3. `test_cmis_profile_gates_visibility_by_access_point` has the same root
+   `/children` expectation for admin/export visibility.
+
+The navigation failures are contract alignment issues: either root children
+should expose documents directly for the CMIS profile, or the tests should ask
+for the folder that contains those projected documents. The OpenAPI failure is
+a concrete implementation bug.
+
+## Native Route Probe Result
+
+Direct live probes confirmed useful native behavior:
+
+- `/cmis` lists multiple profiled access points.
+- `/cmis/compat-tck/browser` returns repository metadata and conservative
+  unsupported-feature diagnostics.
+- `/cmis/compat-tck/browser/types` returns six type projections.
+- `/cmis/compat-tck/browser/children` returns synthetic root folders.
+- `/cmis/compat-tck/browser/query?q=SELECT * FROM cmis:document` returns
+  document projections.
+- `/cmis/compat-tck/browser/object/{object_id}` returns object properties,
+  content stream metadata, allowable actions, version projection, and ACL
+  projection.
+- `/cmis/compat-tck/browser/content-bytes/{object_id}` returns real
+  `text/markdown` bytes.
+- `/cmis/compat-tck/browser/parents/{object_id}` returns projection-only parent
+  folders.
+- Unsupported query/property-update cases return structured diagnostics.
+
+This confirms that content-stream and object-projection foundations exist. The
+OpenCMIS blocker is the external CMIS Browser Binding adapter contract.
+
+## Capability Interpretation
+
+Evidence-backed commodity CMIS compatibility is currently lower than the
+pre-TCK estimate:
+
+- Broad OpenCMIS client compatibility: `pre-compliance`, approximately
+  `0-10%` until session creation succeeds.
+- Harness maturity score from the live run: `9.52`.
+- Controlled-client usefulness of the native CMIS-inspired routes remains
+  materially higher, approximately `55-60%`, because known clients can use the
+  documented JSON routes and profile semantics.
+
+The previous `docs/cmis-1-1-capability-scorecard.md` estimate remains useful as
+a product-depth hypothesis, but it must not be presented as external CMIS
+compatibility evidence until OpenCMIS can connect and execute test cases.
+
+## Findings
+
+| ID | Severity | Finding | Evidence | Required response |
+| --- | --- | --- | --- | --- |
+| CMIS-TCK-20260508-001 | Critical | OpenCMIS cannot create a Browser Binding session because repository info is invalid for CMIS expectations. | ConsoleRunner: `Found invalid Repository Info!` | Implement CMIS-shaped service document/repository info and rerun repository/type group. |
+| CMIS-TCK-20260508-002 | High | The endpoint and TCK target disagree on repository ID. | Endpoint reports `kontextual-compat-tck`; TCK profile uses `compat-tck`. | Align repository ID or provide a stable alias before rerunning TCK. |
+| CMIS-TCK-20260508-003 | High | Browser Binding response fields use snake_case/native DTO shape instead of CMIS Browser Binding shape. | Preflight top-level keys and capability keys are snake_case. | Add CMIS protocol serializers that translate from native DTOs to CMIS Browser Binding JSON. |
+| CMIS-TCK-20260508-004 | Medium | `/openapi.json` generation fails for the content byte streaming route. | Focused internal CMIS suite failure in `test_cmis_browser_binding_routes_are_advertised_in_openapi`. | Fix FastAPI route metadata for `StreamingResponse`. |
+| CMIS-TCK-20260508-005 | Medium | Root `/children` contract is ambiguous between synthetic-folder discovery and direct document listing. | Focused internal CMIS suite failures for readonly and admin/export children expectations. | Decide CMIS root behavior and align implementation/tests with Browser Binding expectations. |
+| CMIS-TCK-20260508-006 | Medium | The TCK facility reports target protocol failures as generic `infrastructure_error`. | Normalized findings classify both TCK groups as infrastructure errors. | Improve classifier/preflight in `open-cmis-tck` so invalid repository info is reported as target protocol shape failure. |
+
+## Recommended Fix Sequence
+
+1. Define exact CMIS Browser Binding service-root/repository-info examples based
+   on OpenCMIS expectations.
+2. Implement a separate CMIS protocol serialization layer rather than exposing
+   native snake_case DTOs directly.
+3. Align `compat-tck` repository ID and target profile.
+4. Make preflight reject native-shaped repository info before invoking Maven.
+5. Fix OpenAPI generation for streaming routes.
+6. Resolve root navigation semantics and add explicit tests for root,
+   synthetic folders, and object-by-path behavior.
+7. Rerun only the repository/type group until OpenCMIS session creation passes.
+8. Then enable object/content TCK and work through semantic failures.
+9. Update the public scorecard only from TCK-backed evidence.
+
+## Boundary
+
+This assessment does not claim CMIS certification or full conformance. It is a
+local engineering assessment using Apache Chemistry OpenCMIS TCK as the first
+external compatibility oracle.
--- a/workplans/KONT-WP-0013-cmis-browser-binding-tck-compatibility.md
+++ b/workplans/KONT-WP-0013-cmis-browser-binding-tck-compatibility.md
@@ -0,0 +1,249 @@
+---
+id: KONT-WP-0013
+type: workplan
+title: "CMIS Browser Binding TCK Compatibility"
+domain: markitect
+repo: kontextual-engine
+status: planned
+owner: codex
+topic_slug: markitect
+planning_priority: high
+planning_order: 13
+created: "2026-05-08"
+updated: "2026-05-08"
+state_hub_workstream_id: "18710625-e573-45b1-9b0a-a8b43caaada1"
+---
+
+# KONT-WP-0013: CMIS Browser Binding TCK Compatibility
+
+## Purpose
+
+Turn the current CMIS-inspired adapter into a CMIS 1.1 Browser Binding surface
+that Apache Chemistry OpenCMIS can connect to and assess. The immediate goal is
+not full CMIS certification; it is to pass session creation and then drive
+repository/type plus object/content behavior from real TCK evidence.
+
+## Trigger
+
+The 2026-05-08 live `guide-board` + `open-cmis-tck` run reached the endpoint,
+but OpenCMIS failed before semantic test cases could execute:
+
+```text
+Found invalid Repository Info!
+```
+
+The result is persisted in
+`docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md`.
+
+## References
+
+- `docs/cmis-opencmis-tck-assessment-2026-05-08T063312Z.md`
+- `docs/cmis-1-1-capability-scorecard.md`
+- `docs/cmis-compliance-assessment.md`
+- `docs/cmis-profiled-access-points-implementation.md`
+- `src/kontextual_engine/api/app.py`
+- `src/kontextual_engine/core/cmis.py`
+- `tests/cmis/test_cmis_browser_binding_api.py`
+- `tests/cmis/test_cmis_runtime_browser_binding.py`
+- `/home/worsch/open-cmis-tck/profiles/targets/kontextual-cmis-compat.json`
+- `/home/worsch/open-cmis-tck/profiles/assessments/cmis-browser-baseline.json`
+
+## Boundary
+
+This workplan fixes Browser Binding compatibility for the profiled CMIS
+connector. It must not turn CMIS into a second domain model. Native engine DTOs,
+policy, blob handling, audit, and repository services remain authoritative; the
+CMIS layer translates those capabilities into standard protocol shapes.
+
+AtomPub and Web Services remain unsupported by design. Mutating multifiling,
+PWC/versioning services, broad CMIS SQL, ACL mutation, policies, renditions,
+retention/hold, and bulk updates remain out of scope unless a later workplan
+adds them explicitly.
+
+## Desired Outcome
+
+- OpenCMIS Browser Binding session creation succeeds against `compat-tck`.
+- Repository/type TCK group executes and produces parsed cases instead of an
+  infrastructure/protocol blocker.
+- Object/content TCK group executes far enough to expose true semantic gaps.
+- Internal CMIS route tests pass with service dependencies installed.
+- Updated CMIS scorecard separates TCK-backed compatibility from native
+  controlled-client usefulness.
+
+## F13.1 - Define Browser Binding protocol contract examples
+
+```task
+id: KONT-WP-0013-T001
+status: todo
+priority: high
+state_hub_task_id: "fa2f99fb-51ed-4fba-8f36-ba4b790908e5"
+```
+
+Acceptance:
+
+- Repository service document/repository-info examples are documented in the
+  exact shape expected by OpenCMIS Browser Binding.
+- Required field names, capability names, repository ID handling, root folder
+  ID handling, and service URLs are covered.
+- The examples include at least one valid `compat-tck` repository response and
+  one invalid native-shaped response used as a negative preflight fixture.
+
+## F13.2 - Implement CMIS Browser Binding serializers
+
+```task
+id: KONT-WP-0013-T002
+status: todo
+priority: high
+state_hub_task_id: "39949f5b-32bb-4ec7-b0f6-83a16ce443b7"
+```
+
+Acceptance:
+
+- Repository info, capabilities, type definitions, object envelopes, property
+  maps, content stream descriptors, ACL projections, and change entries are
+  emitted through CMIS protocol serializers.
+- Native snake_case DTOs are no longer exposed directly on Browser Binding
+  compatibility routes.
+- Serializer tests assert CMIS field names such as `repositoryId`,
+  `repositoryName`, `cmisVersionSupported`, and `rootFolderId`.
+
+## F13.3 - Align target repository identity and access-point profile
+
+```task
+id: KONT-WP-0013-T003
+status: todo
+priority: high
+state_hub_task_id: "96e11fe6-68c6-46ce-88b8-bda0b9baf2d5"
+```
+
+Acceptance:
+
+- The `compat-tck` access point and `open-cmis-tck` target profile agree on the
+  repository ID.
+- If an alias is needed, it is explicit, deterministic, and documented.
+- The port/profile assumptions for local TCK runs are documented without
+  relying on accidental services already bound to port `8000`.
+
+## F13.4 - Add protocol-shape preflight gates
+
+```task
+id: KONT-WP-0013-T004
+status: todo
+priority: high
+state_hub_task_id: "06506455-b3c3-4f17-ae8a-8ffebc226f73"
+```
+
+Acceptance:
+
+- `open-cmis-tck` preflight fails native-shaped repository info before invoking
+  Maven.
+- The failure is classified as target protocol shape incompatibility, not a
+  generic infrastructure error.
+- The preflight reports actionable missing or misshaped fields.
+
+## F13.5 - Fix OpenAPI generation for streaming routes
+
+```task
+id: KONT-WP-0013-T005
+status: todo
+priority: medium
+state_hub_task_id: "de73a3ff-ddf9-40bb-b89c-50d6fabdb841"
+```
+
+Acceptance:
+
+- `/openapi.json` renders successfully with CMIS byte-stream routes enabled.
+- The route still returns a real streaming response at runtime.
+- `tests/cmis/test_cmis_browser_binding_api.py::test_cmis_browser_binding_routes_are_advertised_in_openapi`
+  passes with service dependencies installed.
+
+## F13.6 - Resolve navigation contract and root children behavior
+
+```task
+id: KONT-WP-0013-T006
+status: todo
+priority: medium
+state_hub_task_id: "1cf8d36c-6127-46cd-b409-dda7a045b8dd"
+```
+
+Acceptance:
+
+- Root folder behavior is explicitly defined for Browser Binding compatibility.
+- Tests cover root children, synthetic folder children, projected parents, and
+  hidden object filtering.
+- Internal route tests and OpenCMIS expectations do not disagree about whether
+  root returns only folders or direct documents.
+
+## F13.7 - Establish OpenCMIS session smoke test
+
+```task
+id: KONT-WP-0013-T007
+status: todo
+priority: high
+state_hub_task_id: "38777aea-7d2d-4b30-99b4-5194f2c2f8b0"
+```
+
+Acceptance:
+
+- A fast smoke check proves that OpenCMIS can list repositories and retrieve
+  repository info before full TCK groups run.
+- The smoke check is captured by `guide-board`/`open-cmis-tck` artifacts.
+- Failure messages preserve the underlying OpenCMIS exception.
+
+## F13.8 - Rerun repository/type TCK group to parsed-case evidence
+
+```task
+id: KONT-WP-0013-T008
+status: todo
+priority: high
+state_hub_task_id: "8e7b2235-d39a-465a-b828-b05eb6bcfce3"
+```
+
+Acceptance:
+
+- `repository-type` no longer exits at session creation.
+- Basics and Types test cases are parsed and mapped into normalized evidence.
+- Any semantic failures become concrete follow-up tasks with capability labels.
+
+## F13.9 - Rerun object/content TCK group and classify gaps
+
+```task
+id: KONT-WP-0013-T009
+status: todo
+priority: high
+state_hub_task_id: "22af3e39-c217-4f6b-b54a-5ecea3894020"
+```
+
+Acceptance:
+
+- `object-content` reaches parsed TCK cases.
+- Failures are separated into supported-fix, unsupported-by-design, and profile
+  expectation categories.
+- Content stream read/write behavior is tested against both internal route
+  tests and OpenCMIS expectations.
+
+## F13.10 - Update scorecard and compatibility documentation
+
+```task
+id: KONT-WP-0013-T010
+status: todo
+priority: medium
+state_hub_task_id: "469a3ea6-a6ab-4e6e-ba4f-5b1abdabd6c4"
+```
+
+Acceptance:
+
+- `docs/cmis-1-1-capability-scorecard.md` is updated from TCK-backed evidence.
+- The scorecard separates broad CMIS client compatibility from controlled
+  native-profile usefulness.
+- Deployment docs explain which CMIS clients are expected to work and which
+  capabilities are honestly unsupported.
+
+## Definition Of Done
+
+- Focused internal CMIS tests pass with service dependencies installed.
+- OpenCMIS can establish a Browser Binding session against `compat-tck`.
+- Repository/type TCK group produces parsed evidence.
+- Object/content TCK group produces parsed evidence or documented
+  unsupported-by-design gaps.
+- Updated documentation clearly states the evidence-backed CMIS posture.