From 4a7d04b835501508eae6f25c40ac3895c2a73e04 Mon Sep 17 00:00:00 2001 From: tegwick Date: Thu, 14 May 2026 02:51:24 +0200 Subject: [PATCH] Assessment run for latest improvements --- docs/cmis-1-1-capability-scorecard.md | 16 +-- ...e-readiness-evidence-2026-05-14T003705Z.md | 111 ++++++++++++++++++ docs/cmis-read-side-contract.md | 26 +++- docs/first-release-readiness.md | 4 +- src/kontextual_engine/api/app.py | 10 +- tests/cmis/test_cmis_browser_binding_api.py | 2 + .../cmis/test_cmis_runtime_browser_binding.py | 2 + ...P-0016-cmis-read-side-contract-maturity.md | 34 ++++-- 8 files changed, 184 insertions(+), 21 deletions(-) create mode 100644 docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md diff --git a/docs/cmis-1-1-capability-scorecard.md b/docs/cmis-1-1-capability-scorecard.md index d7d29a2..fa8e2dd 100644 --- a/docs/cmis-1-1-capability-scorecard.md +++ b/docs/cmis-1-1-capability-scorecard.md @@ -2,12 +2,12 @@ Date: 2026-05-14 -Evidence update: the 2026-05-14 release-warning pass completed the selected -Browser Binding `repository-type` and `object-content` baseline. The latest -run, `run-20260513T223537Z`, reports `0` unexpected findings. The -`object-content` group now passes without warnings; the only remaining -OpenCMIS warning is the local harness using HTTP rather than HTTPS. See -`docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-13T223537Z.md`. +Evidence update: the 2026-05-14 post-WP-0016 release-readiness pass completed +the selected Browser Binding `repository-type` and `object-content` baseline. +The latest run, `run-20260514T003705Z`, reports `0` unexpected findings. The +`object-content` group passes without warnings; the only remaining OpenCMIS +warning is the local harness using HTTP rather than HTTPS. See +`docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md`. The score below remains a product-depth estimate against mature CMIS products. The selected OpenCMIS baseline is now stable preparation evidence for @@ -16,7 +16,9 @@ repository/type and object/content services, not a full CMIS certification. Read-side contract update: `KONT-WP-0016` adds a documented bounded query subset, common CMIS `ORDER BY`, target/either relationship filters, enriched relationship and ACL projections, and explicit `notSupported` diagnostics for -unsupported navigation selectors. +unsupported navigation selectors. The release-readiness rerun also confirms +that child navigation is ordered deterministically by `cmis:name` when +`capabilityOrderBy=common` is advertised. Status: baseline scorecard for the current Browser Binding subset. diff --git a/docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md b/docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md new file mode 100644 index 0000000..6691b98 --- /dev/null +++ b/docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md @@ -0,0 +1,111 @@ +# CMIS OpenCMIS TCK Evidence - Release Readiness - 2026-05-14T00:37:05Z + +## Run Summary + +- Run ID: `run-20260514T003705Z` +- Local date: 2026-05-14 Europe/Berlin +- Harness: `guide-board` with `open-cmis-tck` extension +- Assessment: `cmis-browser-baseline` +- Target: `kontextual-cmis-compat` +- Endpoint: `http://127.0.0.1:8010/cmis/compat-tck/browser` +- OpenCMIS TCK: CMIS 1.1.0, revision `1789681` +- Result: `completed` +- Policy: `0` unexpected findings, `0` applied waivers +- Run directory: `/tmp/kontextual-cmis-assessment-20260514T003648Z` +- Assessment package: + `/tmp/kontextual-cmis-assessment-20260514T003648Z/reports/assessment-package.json` +- Report: + `/tmp/kontextual-cmis-assessment-20260514T003648Z/reports/report.md` + +This run was executed after `KONT-WP-0016` added bounded read-side query, +relationship, ACL, and capability-ordering contracts. A first assessment run +(`run-20260514T002933Z`) exposed an avoidable object/content warning because +folder children were not returned in deterministic `cmis:name` order after +advertising `capabilityOrderBy=common`. The engine now sorts CMIS child +projections by `cmis:name`; the final run below is the persisted release +evidence. + +## Command + +The sister `open-cmis-tck` target profile points at port `8000`, which is not +safe for this workstation because another local service is already bound there. +For this run, the target profile was copied to `/tmp` with only the endpoint +changed to port `8010`. + +```sh +cd /home/worsch/guide-board +source /home/worsch/open-cmis-tck/.local/toolchains/env.sh +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + run \ + --target /tmp/kontextual-cmis-compat-8010-20260514T003648Z.json \ + --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \ + --output-dir /tmp/kontextual-cmis-assessment-20260514T003648Z + +cd /home/worsch/open-cmis-tck +PYTHONPATH=src python3 scripts/cmis_scorecard.py \ + --run-dir /tmp/kontextual-cmis-assessment-20260514T003648Z +``` + +## Normalized Results + +| Group | Result | Counts | Remaining non-green findings | +| --- | --- | --- | --- | +| `preflight` | `pass` | Endpoint reachable; parseable Browser Binding JSON; repository `compat-tck` selected. | None. | +| `repository-type` | `warning` | `38 pass`, `2 info`, `1 skipped`, `1 warning` | Local loopback endpoint uses HTTP rather than HTTPS. | +| `object-content` | `pass` | `10 info`, `5 skipped` | None. | + +Guide Board summary: + +- `pass`: 2 +- `warning`: 1 +- unexpected findings: 0 +- applied expectations: 0 +- applied waivers: 0 + +The only warning is the known local harness transport warning: + +```text +Security Test (BROWSER): HTTPS is not used. Credentials might be transferred as plain text! +``` + +## Score + +This is a compatibility-infrastructure score for the selected Browser Binding +baseline, not a CMIS certification score. + +| Metric | Score | Basis | +| --- | ---: | --- | +| Selected baseline completion | 100.0% | Guide Board result `completed`; both selected TCK groups returned `0`. | +| Unexpected finding clearance | 100.0% | `0` unexpected findings, `0` fail, `0` infrastructure_error. | +| Warning-adjusted normalized case score | 99.1% | `(56 accepted + 0.5 * 1 warning) / 57 normalized cases`. | +| Strict no-warning normalized case score | 98.2% | `56 accepted / 57 normalized cases`. | +| Guide Board maturity score | 33.33 | Two of nine mapped capability groups assessed; object/content demonstrated, repository/type partial due local HTTP. | +| Guide Board coverage | 22.22% | Selected baseline covers `repository-type` and `object-content` only. | + +Digest versus `run-20260514T002933Z`: + +- `object-content` improved from warning to pass. +- The folder child-order warning is closed by deterministic `cmis:name` + ordering. +- The remaining warning is unchanged: local loopback HTTP rather than HTTPS. + +Digest versus `run-20260513T223537Z`: + +- Selected-baseline infrastructure score remains `99.1%`. +- `object-content` remains pass. +- The persisted baseline now reflects `capabilityOrderBy=common`. + +## Interpretation + +The selected OpenCMIS Browser Binding baseline remains release-ready for a +controlled `0.1.0` preview. The engine demonstrates stable repository/type and +object/content behavior through the external OpenCMIS harness. + +The remaining warning is a deployment topology issue, not a CMIS adapter +behavior failure. Released access points must run behind HTTPS termination; +local loopback harness runs may accept the warning as an environment condition. + +This evidence still does not claim full CMIS 1.1 certification. It does not +cover AtomPub, Web Services, PWC/checkin/checkout, full CMIS SQL, renditions, +retention/hold, policy mutation, or the non-selected OpenCMIS TCK groups. diff --git a/docs/cmis-read-side-contract.md b/docs/cmis-read-side-contract.md index 6eb29de..38f304f 100644 --- a/docs/cmis-read-side-contract.md +++ b/docs/cmis-read-side-contract.md @@ -58,6 +58,8 @@ the supported grammar and field sets included. - `capabilityQuery` remains `metadataonly`. - `capabilityOrderBy` is now `common`, not `none`, because common CMIS property ordering is implemented and covered by tests. +- Folder and document children are returned in deterministic `cmis:name` order + to match the common-ordering claim. - `capabilityGetDescendants` and `capabilityGetFolderTree` remain false. - Multifiling remains projection-only. Mutation semantics are deliberately out of scope for this read-side contract. @@ -79,12 +81,17 @@ python3 -m pytest \ Result: 21 passed, 16 skipped in 5.19s ``` -Browser Binding API verification with optional service extras: +Focused CMIS verification with optional service extras: ```text -.venv/bin/python -m pytest tests/cmis/test_cmis_browser_binding_api.py -q +.venv/bin/python -m pytest \ + tests/cmis/test_cmis_browser_binding_api.py \ + tests/cmis/test_cmis_runtime_browser_binding.py \ + tests/cmis/test_cmis_compliance_flags.py \ + tests/cmis/test_cmis_contract_examples.py \ + -q -Result: 16 passed in 39.09s +Result: 37 passed in 44.72s ``` The default system-Python run skips Browser Binding API tests when FastAPI/HTTPX @@ -92,3 +99,16 @@ are unavailable and skips capacity probes unless `KONTEXTUAL_RUN_CAPACITY=1` is set. The capacity probe exercises 400 documents and 250 relationships over query and target-filter paths, while relying on the shared performance history monitor for drift tracking. + +OpenCMIS selected-baseline verification: + +```text +run-20260514T003705Z +cmis-browser-baseline: completed +Guide Board summary: 2 pass, 1 warning +Policy: 0 unexpected findings +``` + +The only remaining OpenCMIS warning is local HTTP transport on the loopback +harness. `object-content` passes after child projections were made +deterministically ordered by `cmis:name`. diff --git a/docs/first-release-readiness.md b/docs/first-release-readiness.md index 3d2a3f6..cff0b4b 100644 --- a/docs/first-release-readiness.md +++ b/docs/first-release-readiness.md @@ -30,8 +30,8 @@ Out of scope for `0.1.0`: | Area | Gate | Current state | | --- | --- | --- | | CMIS read-side contract | Query, navigation, relationship, ACL, and change-token contracts are release-stable or explicitly waived. | `KONT-WP-0016` implemented locally; full release verification still required. | -| Tests | Full suite passes in the project venv. | `.venv/bin/python -m pytest -q` passed: `166 passed`, `15 skipped`; advisory performance drift warnings recorded. | -| CMIS evidence | OpenCMIS selected baseline completes with no unexpected findings. | `run-20260513T223537Z` completed; only local HTTP warning remains. | +| Tests | Full suite passes in the project venv. | `.venv/bin/python -m pytest -q` passed: `166 passed`, `15 skipped` in `73.05s`; advisory performance drift warnings recorded. | +| CMIS evidence | OpenCMIS selected baseline completes with no unexpected findings. | `run-20260514T003705Z` completed; object/content passed; only local HTTP warning remains. | | Transport | Released CMIS access points are served behind HTTPS. | Required deployment gate; local loopback warning is accepted only for harness runs. | | Capability honesty | Scorecard, unsupported catalog, and examples match behavior. | Updated for `appendContentStream` and WP-0016 read-side contract; final doc review required. | | Packaging | Version, dependencies, optional extras, and install smoke are checked. | `pyproject.toml` is already `0.1.0`; build/install smoke still required. | diff --git a/src/kontextual_engine/api/app.py b/src/kontextual_engine/api/app.py index 37c0bde..7e7d500 100644 --- a/src/kontextual_engine/api/app.py +++ b/src/kontextual_engine/api/app.py @@ -1801,7 +1801,7 @@ class ServiceRuntime: for path in sorted(child_folder_paths) if path not in workspace_folders ] - return sorted(workspace_children + projection_children, key=lambda item: item["path"]) + return sorted(workspace_children + projection_children, key=_cmis_child_order_key) children: list[dict[str, Any]] = [] folder_path = _normalize_cmis_path(folder_path) child_folder_paths: set[str] = set() @@ -1833,7 +1833,7 @@ class ServiceRuntime: for path in sorted(child_folder_paths) if path not in workspace_folders ] - return projection_children + children + return sorted(projection_children + children, key=_cmis_child_order_key) def _cmis_workspace_folder_map(self, access_point_id: str) -> dict[str, CMISWorkspaceFolder]: return self.cmis_workspace_folders.setdefault(access_point_id, {}) @@ -5050,6 +5050,12 @@ def _path_contains(parent_path: str, candidate_path: str) -> bool: return candidate == parent or candidate.startswith(parent.rstrip("/") + "/") +def _cmis_child_order_key(item: dict[str, Any]) -> tuple[str, str]: + properties = dict(item.get("properties", {})) + name = properties.get("cmis:name") or item.get("name") or _path_name(str(item.get("path") or "")) + return (str(name).casefold(), str(item.get("object_id") or "")) + + def _parse_multipart_form(content_type: str, body: bytes) -> tuple[dict[str, list[Any]], dict[str, dict[str, Any]]]: message = BytesParser(policy=policy.default).parsebytes( b"Content-Type: " + content_type.encode("utf-8") + b"\r\nMIME-Version: 1.0\r\n\r\n" + body diff --git a/tests/cmis/test_cmis_browser_binding_api.py b/tests/cmis/test_cmis_browser_binding_api.py index 8e569c0..8377d16 100644 --- a/tests/cmis/test_cmis_browser_binding_api.py +++ b/tests/cmis/test_cmis_browser_binding_api.py @@ -183,10 +183,12 @@ def test_cmis_readonly_children_object_content_query_relationships_and_changes(c root_ids = {item["object_id"] for item in root_children["objects"]} child_ids = {item["object_id"] for item in children["objects"]} + child_names = [item["name"] for item in children["objects"]] assert "cmis:folder:assets" in root_ids assert "cmis:asset:asset-source" in child_ids assert "cmis:asset:asset-public" in child_ids assert "cmis:asset:asset-confidential" not in child_ids + assert child_names == sorted(child_names, key=str.casefold) assert object_response["properties"]["kontextual:assetId"] == "asset-source" assert "get_content_stream" in object_response["allowable_actions"] assert content["mime_type"] == "text/markdown" diff --git a/tests/cmis/test_cmis_runtime_browser_binding.py b/tests/cmis/test_cmis_runtime_browser_binding.py index 92f86c0..c37249c 100644 --- a/tests/cmis/test_cmis_runtime_browser_binding.py +++ b/tests/cmis/test_cmis_runtime_browser_binding.py @@ -96,9 +96,11 @@ def test_runtime_cmis_browser_repository_types_children_and_object(cmis_runtime) assert repository["unsupported_features"]["get_descendants"]["status"] == "unsupported" assert {item["base_type_id"] for item in types["items"]} >= {"cmis:document", "cmis:folder"} root_paths = {item["path"] for item in children["objects"]} + child_names = [item["name"] for item in topic_children["objects"]] topic_object_ids = {item["object_id"] for item in topic_children["objects"]} parent_paths = {item["path"] for item in parents["parents"]} assert "/topics" in root_paths + assert child_names == sorted(child_names, key=str.casefold) assert "cmis:asset:asset-runtime-source" in topic_object_ids assert "cmis:asset:asset-runtime-confidential" not in topic_object_ids assert {"/topics/cmis", "/topics/integration"} <= parent_paths diff --git a/workplans/KONT-WP-0016-cmis-read-side-contract-maturity.md b/workplans/KONT-WP-0016-cmis-read-side-contract-maturity.md index cfb3881..181456b 100644 --- a/workplans/KONT-WP-0016-cmis-read-side-contract-maturity.md +++ b/workplans/KONT-WP-0016-cmis-read-side-contract-maturity.md @@ -265,16 +265,19 @@ Result: 21 passed, 16 skipped in 5.19s The skipped tests require optional FastAPI/HTTPX test extras in this local environment. They remain part of the release verification gate when the service -extras are installed. No new OpenCMIS run was required for the selected -object/content baseline because this workplan changes read-side query, -relationship, ACL, and diagnostics contracts outside that selected baseline. +extras are installed. -Browser Binding API verification with service extras: +Focused CMIS verification with service extras: ```text -.venv/bin/python -m pytest tests/cmis/test_cmis_browser_binding_api.py -q +.venv/bin/python -m pytest \ + tests/cmis/test_cmis_browser_binding_api.py \ + tests/cmis/test_cmis_runtime_browser_binding.py \ + tests/cmis/test_cmis_compliance_flags.py \ + tests/cmis/test_cmis_contract_examples.py \ + -q -Result: 16 passed in 39.09s +Result: 37 passed in 44.72s ``` Full suite verification with service extras: @@ -282,13 +285,30 @@ Full suite verification with service extras: ```text .venv/bin/python -m pytest -q -Result: 166 passed, 15 skipped in 55.94s +Result: 166 passed, 15 skipped in 73.05s ``` The run emitted advisory performance-drift warnings for several API tests. They do not indicate functional failures and should be watched through the existing compact performance-history monitor. +OpenCMIS release-readiness assessment: + +```text +run-20260514T003705Z +Assessment: cmis-browser-baseline +Result: completed +Guide Board summary: 2 pass, 1 warning +Policy: 0 unexpected findings, 0 applied waivers +``` + +The first post-WP-0016 assessment run exposed an avoidable `cmis:name` child +ordering warning after `capabilityOrderBy=common` was advertised. CMIS children +are now sorted deterministically by `cmis:name`; the final persisted run has +`object-content` passing and only the known local HTTP transport warning +remaining. Evidence is persisted in +`docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md`. + ## Release Advice This workplan should run before `KONT-WP-0015` unless the first release is