Assessment run for latest improvements

2026-05-14 02:51:24 +02:00
parent e5197e15e2
commit 4a7d04b835
8 changed files with 184 additions and 21 deletions
--- a/docs/cmis-1-1-capability-scorecard.md
+++ b/docs/cmis-1-1-capability-scorecard.md
@@ -2,12 +2,12 @@

 Date: 2026-05-14

-Evidence update: the 2026-05-14 release-warning pass completed the selected
-Browser Binding `repository-type` and `object-content` baseline. The latest
-run, `run-20260513T223537Z`, reports `0` unexpected findings. The
-`object-content` group now passes without warnings; the only remaining
-OpenCMIS warning is the local harness using HTTP rather than HTTPS. See
-`docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-13T223537Z.md`.
+Evidence update: the 2026-05-14 post-WP-0016 release-readiness pass completed
+the selected Browser Binding `repository-type` and `object-content` baseline.
+The latest run, `run-20260514T003705Z`, reports `0` unexpected findings. The
+`object-content` group passes without warnings; the only remaining OpenCMIS
+warning is the local harness using HTTP rather than HTTPS. See
+`docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md`.

 The score below remains a product-depth estimate against mature CMIS products.
 The selected OpenCMIS baseline is now stable preparation evidence for
@@ -16,7 +16,9 @@ repository/type and object/content services, not a full CMIS certification.
 Read-side contract update: `KONT-WP-0016` adds a documented bounded query
 subset, common CMIS `ORDER BY`, target/either relationship filters, enriched
 relationship and ACL projections, and explicit `notSupported` diagnostics for
-unsupported navigation selectors.
+unsupported navigation selectors. The release-readiness rerun also confirms
+that child navigation is ordered deterministically by `cmis:name` when
+`capabilityOrderBy=common` is advertised.

 Status: baseline scorecard for the current Browser Binding subset.

--- a/docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md
+++ b/docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md
@@ -0,0 +1,111 @@
+# CMIS OpenCMIS TCK Evidence - Release Readiness - 2026-05-14T00:37:05Z
+
+## Run Summary
+
+- Run ID: `run-20260514T003705Z`
+- Local date: 2026-05-14 Europe/Berlin
+- Harness: `guide-board` with `open-cmis-tck` extension
+- Assessment: `cmis-browser-baseline`
+- Target: `kontextual-cmis-compat`
+- Endpoint: `http://127.0.0.1:8010/cmis/compat-tck/browser`
+- OpenCMIS TCK: CMIS 1.1.0, revision `1789681`
+- Result: `completed`
+- Policy: `0` unexpected findings, `0` applied waivers
+- Run directory: `/tmp/kontextual-cmis-assessment-20260514T003648Z`
+- Assessment package:
+  `/tmp/kontextual-cmis-assessment-20260514T003648Z/reports/assessment-package.json`
+- Report:
+  `/tmp/kontextual-cmis-assessment-20260514T003648Z/reports/report.md`
+
+This run was executed after `KONT-WP-0016` added bounded read-side query,
+relationship, ACL, and capability-ordering contracts. A first assessment run
+(`run-20260514T002933Z`) exposed an avoidable object/content warning because
+folder children were not returned in deterministic `cmis:name` order after
+advertising `capabilityOrderBy=common`. The engine now sorts CMIS child
+projections by `cmis:name`; the final run below is the persisted release
+evidence.
+
+## Command
+
+The sister `open-cmis-tck` target profile points at port `8000`, which is not
+safe for this workstation because another local service is already bound there.
+For this run, the target profile was copied to `/tmp` with only the endpoint
+changed to port `8010`.
+
+```sh
+cd /home/worsch/guide-board
+source /home/worsch/open-cmis-tck/.local/toolchains/env.sh
+PYTHONPATH=src python3 -m guide_board \
+  --extension-dir ../open-cmis-tck \
+  run \
+  --target /tmp/kontextual-cmis-compat-8010-20260514T003648Z.json \
+  --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \
+  --output-dir /tmp/kontextual-cmis-assessment-20260514T003648Z
+
+cd /home/worsch/open-cmis-tck
+PYTHONPATH=src python3 scripts/cmis_scorecard.py \
+  --run-dir /tmp/kontextual-cmis-assessment-20260514T003648Z
+```
+
+## Normalized Results
+
+| Group | Result | Counts | Remaining non-green findings |
+| --- | --- | --- | --- |
+| `preflight` | `pass` | Endpoint reachable; parseable Browser Binding JSON; repository `compat-tck` selected. | None. |
+| `repository-type` | `warning` | `38 pass`, `2 info`, `1 skipped`, `1 warning` | Local loopback endpoint uses HTTP rather than HTTPS. |
+| `object-content` | `pass` | `10 info`, `5 skipped` | None. |
+
+Guide Board summary:
+
+- `pass`: 2
+- `warning`: 1
+- unexpected findings: 0
+- applied expectations: 0
+- applied waivers: 0
+
+The only warning is the known local harness transport warning:
+
+```text
+Security Test (BROWSER): HTTPS is not used. Credentials might be transferred as plain text!
+```
+
+## Score
+
+This is a compatibility-infrastructure score for the selected Browser Binding
+baseline, not a CMIS certification score.
+
+| Metric | Score | Basis |
+| --- | ---: | --- |
+| Selected baseline completion | 100.0% | Guide Board result `completed`; both selected TCK groups returned `0`. |
+| Unexpected finding clearance | 100.0% | `0` unexpected findings, `0` fail, `0` infrastructure_error. |
+| Warning-adjusted normalized case score | 99.1% | `(56 accepted + 0.5 * 1 warning) / 57 normalized cases`. |
+| Strict no-warning normalized case score | 98.2% | `56 accepted / 57 normalized cases`. |
+| Guide Board maturity score | 33.33 | Two of nine mapped capability groups assessed; object/content demonstrated, repository/type partial due local HTTP. |
+| Guide Board coverage | 22.22% | Selected baseline covers `repository-type` and `object-content` only. |
+
+Digest versus `run-20260514T002933Z`:
+
+- `object-content` improved from warning to pass.
+- The folder child-order warning is closed by deterministic `cmis:name`
+  ordering.
+- The remaining warning is unchanged: local loopback HTTP rather than HTTPS.
+
+Digest versus `run-20260513T223537Z`:
+
+- Selected-baseline infrastructure score remains `99.1%`.
+- `object-content` remains pass.
+- The persisted baseline now reflects `capabilityOrderBy=common`.
+
+## Interpretation
+
+The selected OpenCMIS Browser Binding baseline remains release-ready for a
+controlled `0.1.0` preview. The engine demonstrates stable repository/type and
+object/content behavior through the external OpenCMIS harness.
+
+The remaining warning is a deployment topology issue, not a CMIS adapter
+behavior failure. Released access points must run behind HTTPS termination;
+local loopback harness runs may accept the warning as an environment condition.
+
+This evidence still does not claim full CMIS 1.1 certification. It does not
+cover AtomPub, Web Services, PWC/checkin/checkout, full CMIS SQL, renditions,
+retention/hold, policy mutation, or the non-selected OpenCMIS TCK groups.
--- a/docs/cmis-read-side-contract.md
+++ b/docs/cmis-read-side-contract.md
@@ -58,6 +58,8 @@ the supported grammar and field sets included.
 - `capabilityQuery` remains `metadataonly`.
 - `capabilityOrderBy` is now `common`, not `none`, because common CMIS property
  ordering is implemented and covered by tests.
+- Folder and document children are returned in deterministic `cmis:name` order
+  to match the common-ordering claim.
 - `capabilityGetDescendants` and `capabilityGetFolderTree` remain false.
 - Multifiling remains projection-only. Mutation semantics are deliberately out
  of scope for this read-side contract.
@@ -79,12 +81,17 @@ python3 -m pytest \
 Result: 21 passed, 16 skipped in 5.19s
 ```

-Browser Binding API verification with optional service extras:
+Focused CMIS verification with optional service extras:

 ```text
-.venv/bin/python -m pytest tests/cmis/test_cmis_browser_binding_api.py -q
+.venv/bin/python -m pytest \
+  tests/cmis/test_cmis_browser_binding_api.py \
+  tests/cmis/test_cmis_runtime_browser_binding.py \
+  tests/cmis/test_cmis_compliance_flags.py \
+  tests/cmis/test_cmis_contract_examples.py \
+  -q

-Result: 16 passed in 39.09s
+Result: 37 passed in 44.72s
 ```

 The default system-Python run skips Browser Binding API tests when FastAPI/HTTPX
@@ -92,3 +99,16 @@ are unavailable and skips capacity probes unless `KONTEXTUAL_RUN_CAPACITY=1` is
 set. The capacity probe exercises 400 documents and 250 relationships over
 query and target-filter paths, while relying on the shared performance history
 monitor for drift tracking.
+
+OpenCMIS selected-baseline verification:
+
+```text
+run-20260514T003705Z
+cmis-browser-baseline: completed
+Guide Board summary: 2 pass, 1 warning
+Policy: 0 unexpected findings
+```
+
+The only remaining OpenCMIS warning is local HTTP transport on the loopback
+harness. `object-content` passes after child projections were made
+deterministically ordered by `cmis:name`.
--- a/docs/first-release-readiness.md
+++ b/docs/first-release-readiness.md
@@ -30,8 +30,8 @@ Out of scope for `0.1.0`:
 | Area | Gate | Current state |
 | --- | --- | --- |
 | CMIS read-side contract | Query, navigation, relationship, ACL, and change-token contracts are release-stable or explicitly waived. | `KONT-WP-0016` implemented locally; full release verification still required. |
-| Tests | Full suite passes in the project venv. | `.venv/bin/python -m pytest -q` passed: `166 passed`, `15 skipped`; advisory performance drift warnings recorded. |
-| CMIS evidence | OpenCMIS selected baseline completes with no unexpected findings. | `run-20260513T223537Z` completed; only local HTTP warning remains. |
+| Tests | Full suite passes in the project venv. | `.venv/bin/python -m pytest -q` passed: `166 passed`, `15 skipped` in `73.05s`; advisory performance drift warnings recorded. |
+| CMIS evidence | OpenCMIS selected baseline completes with no unexpected findings. | `run-20260514T003705Z` completed; object/content passed; only local HTTP warning remains. |
 | Transport | Released CMIS access points are served behind HTTPS. | Required deployment gate; local loopback warning is accepted only for harness runs. |
 | Capability honesty | Scorecard, unsupported catalog, and examples match behavior. | Updated for `appendContentStream` and WP-0016 read-side contract; final doc review required. |
 | Packaging | Version, dependencies, optional extras, and install smoke are checked. | `pyproject.toml` is already `0.1.0`; build/install smoke still required. |