generated from coulomb/repo-seed
Assessment run for latest improvements
This commit is contained in:
@@ -2,12 +2,12 @@
|
|||||||
|
|
||||||
Date: 2026-05-14
|
Date: 2026-05-14
|
||||||
|
|
||||||
Evidence update: the 2026-05-14 release-warning pass completed the selected
|
Evidence update: the 2026-05-14 post-WP-0016 release-readiness pass completed
|
||||||
Browser Binding `repository-type` and `object-content` baseline. The latest
|
the selected Browser Binding `repository-type` and `object-content` baseline.
|
||||||
run, `run-20260513T223537Z`, reports `0` unexpected findings. The
|
The latest run, `run-20260514T003705Z`, reports `0` unexpected findings. The
|
||||||
`object-content` group now passes without warnings; the only remaining
|
`object-content` group passes without warnings; the only remaining OpenCMIS
|
||||||
OpenCMIS warning is the local harness using HTTP rather than HTTPS. See
|
warning is the local harness using HTTP rather than HTTPS. See
|
||||||
`docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-13T223537Z.md`.
|
`docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md`.
|
||||||
|
|
||||||
The score below remains a product-depth estimate against mature CMIS products.
|
The score below remains a product-depth estimate against mature CMIS products.
|
||||||
The selected OpenCMIS baseline is now stable preparation evidence for
|
The selected OpenCMIS baseline is now stable preparation evidence for
|
||||||
@@ -16,7 +16,9 @@ repository/type and object/content services, not a full CMIS certification.
|
|||||||
Read-side contract update: `KONT-WP-0016` adds a documented bounded query
|
Read-side contract update: `KONT-WP-0016` adds a documented bounded query
|
||||||
subset, common CMIS `ORDER BY`, target/either relationship filters, enriched
|
subset, common CMIS `ORDER BY`, target/either relationship filters, enriched
|
||||||
relationship and ACL projections, and explicit `notSupported` diagnostics for
|
relationship and ACL projections, and explicit `notSupported` diagnostics for
|
||||||
unsupported navigation selectors.
|
unsupported navigation selectors. The release-readiness rerun also confirms
|
||||||
|
that child navigation is ordered deterministically by `cmis:name` when
|
||||||
|
`capabilityOrderBy=common` is advertised.
|
||||||
|
|
||||||
Status: baseline scorecard for the current Browser Binding subset.
|
Status: baseline scorecard for the current Browser Binding subset.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,111 @@
|
|||||||
|
# CMIS OpenCMIS TCK Evidence - Release Readiness - 2026-05-14T00:37:05Z
|
||||||
|
|
||||||
|
## Run Summary
|
||||||
|
|
||||||
|
- Run ID: `run-20260514T003705Z`
|
||||||
|
- Local date: 2026-05-14 Europe/Berlin
|
||||||
|
- Harness: `guide-board` with `open-cmis-tck` extension
|
||||||
|
- Assessment: `cmis-browser-baseline`
|
||||||
|
- Target: `kontextual-cmis-compat`
|
||||||
|
- Endpoint: `http://127.0.0.1:8010/cmis/compat-tck/browser`
|
||||||
|
- OpenCMIS TCK: CMIS 1.1.0, revision `1789681`
|
||||||
|
- Result: `completed`
|
||||||
|
- Policy: `0` unexpected findings, `0` applied waivers
|
||||||
|
- Run directory: `/tmp/kontextual-cmis-assessment-20260514T003648Z`
|
||||||
|
- Assessment package:
|
||||||
|
`/tmp/kontextual-cmis-assessment-20260514T003648Z/reports/assessment-package.json`
|
||||||
|
- Report:
|
||||||
|
`/tmp/kontextual-cmis-assessment-20260514T003648Z/reports/report.md`
|
||||||
|
|
||||||
|
This run was executed after `KONT-WP-0016` added bounded read-side query,
|
||||||
|
relationship, ACL, and capability-ordering contracts. A first assessment run
|
||||||
|
(`run-20260514T002933Z`) exposed an avoidable object/content warning because
|
||||||
|
folder children were not returned in deterministic `cmis:name` order after
|
||||||
|
advertising `capabilityOrderBy=common`. The engine now sorts CMIS child
|
||||||
|
projections by `cmis:name`; the final run below is the persisted release
|
||||||
|
evidence.
|
||||||
|
|
||||||
|
## Command
|
||||||
|
|
||||||
|
The sister `open-cmis-tck` target profile points at port `8000`, which is not
|
||||||
|
safe for this workstation because another local service is already bound there.
|
||||||
|
For this run, the target profile was copied to `/tmp` with only the endpoint
|
||||||
|
changed to port `8010`.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd /home/worsch/guide-board
|
||||||
|
source /home/worsch/open-cmis-tck/.local/toolchains/env.sh
|
||||||
|
PYTHONPATH=src python3 -m guide_board \
|
||||||
|
--extension-dir ../open-cmis-tck \
|
||||||
|
run \
|
||||||
|
--target /tmp/kontextual-cmis-compat-8010-20260514T003648Z.json \
|
||||||
|
--assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \
|
||||||
|
--output-dir /tmp/kontextual-cmis-assessment-20260514T003648Z
|
||||||
|
|
||||||
|
cd /home/worsch/open-cmis-tck
|
||||||
|
PYTHONPATH=src python3 scripts/cmis_scorecard.py \
|
||||||
|
--run-dir /tmp/kontextual-cmis-assessment-20260514T003648Z
|
||||||
|
```
|
||||||
|
|
||||||
|
## Normalized Results
|
||||||
|
|
||||||
|
| Group | Result | Counts | Remaining non-green findings |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `preflight` | `pass` | Endpoint reachable; parseable Browser Binding JSON; repository `compat-tck` selected. | None. |
|
||||||
|
| `repository-type` | `warning` | `38 pass`, `2 info`, `1 skipped`, `1 warning` | Local loopback endpoint uses HTTP rather than HTTPS. |
|
||||||
|
| `object-content` | `pass` | `10 info`, `5 skipped` | None. |
|
||||||
|
|
||||||
|
Guide Board summary:
|
||||||
|
|
||||||
|
- `pass`: 2
|
||||||
|
- `warning`: 1
|
||||||
|
- unexpected findings: 0
|
||||||
|
- applied expectations: 0
|
||||||
|
- applied waivers: 0
|
||||||
|
|
||||||
|
The only warning is the known local harness transport warning:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Security Test (BROWSER): HTTPS is not used. Credentials might be transferred as plain text!
|
||||||
|
```
|
||||||
|
|
||||||
|
## Score
|
||||||
|
|
||||||
|
This is a compatibility-infrastructure score for the selected Browser Binding
|
||||||
|
baseline, not a CMIS certification score.
|
||||||
|
|
||||||
|
| Metric | Score | Basis |
|
||||||
|
| --- | ---: | --- |
|
||||||
|
| Selected baseline completion | 100.0% | Guide Board result `completed`; both selected TCK groups returned `0`. |
|
||||||
|
| Unexpected finding clearance | 100.0% | `0` unexpected findings, `0` fail, `0` infrastructure_error. |
|
||||||
|
| Warning-adjusted normalized case score | 99.1% | `(56 accepted + 0.5 * 1 warning) / 57 normalized cases`. |
|
||||||
|
| Strict no-warning normalized case score | 98.2% | `56 accepted / 57 normalized cases`. |
|
||||||
|
| Guide Board maturity score | 33.33 | Two of nine mapped capability groups assessed; object/content demonstrated, repository/type partial due local HTTP. |
|
||||||
|
| Guide Board coverage | 22.22% | Selected baseline covers `repository-type` and `object-content` only. |
|
||||||
|
|
||||||
|
Digest versus `run-20260514T002933Z`:
|
||||||
|
|
||||||
|
- `object-content` improved from warning to pass.
|
||||||
|
- The folder child-order warning is closed by deterministic `cmis:name`
|
||||||
|
ordering.
|
||||||
|
- The remaining warning is unchanged: local loopback HTTP rather than HTTPS.
|
||||||
|
|
||||||
|
Digest versus `run-20260513T223537Z`:
|
||||||
|
|
||||||
|
- Selected-baseline infrastructure score remains `99.1%`.
|
||||||
|
- `object-content` remains pass.
|
||||||
|
- The persisted baseline now reflects `capabilityOrderBy=common`.
|
||||||
|
|
||||||
|
## Interpretation
|
||||||
|
|
||||||
|
The selected OpenCMIS Browser Binding baseline remains release-ready for a
|
||||||
|
controlled `0.1.0` preview. The engine demonstrates stable repository/type and
|
||||||
|
object/content behavior through the external OpenCMIS harness.
|
||||||
|
|
||||||
|
The remaining warning is a deployment topology issue, not a CMIS adapter
|
||||||
|
behavior failure. Released access points must run behind HTTPS termination;
|
||||||
|
local loopback harness runs may accept the warning as an environment condition.
|
||||||
|
|
||||||
|
This evidence still does not claim full CMIS 1.1 certification. It does not
|
||||||
|
cover AtomPub, Web Services, PWC/checkin/checkout, full CMIS SQL, renditions,
|
||||||
|
retention/hold, policy mutation, or the non-selected OpenCMIS TCK groups.
|
||||||
@@ -58,6 +58,8 @@ the supported grammar and field sets included.
|
|||||||
- `capabilityQuery` remains `metadataonly`.
|
- `capabilityQuery` remains `metadataonly`.
|
||||||
- `capabilityOrderBy` is now `common`, not `none`, because common CMIS property
|
- `capabilityOrderBy` is now `common`, not `none`, because common CMIS property
|
||||||
ordering is implemented and covered by tests.
|
ordering is implemented and covered by tests.
|
||||||
|
- Folder and document children are returned in deterministic `cmis:name` order
|
||||||
|
to match the common-ordering claim.
|
||||||
- `capabilityGetDescendants` and `capabilityGetFolderTree` remain false.
|
- `capabilityGetDescendants` and `capabilityGetFolderTree` remain false.
|
||||||
- Multifiling remains projection-only. Mutation semantics are deliberately out
|
- Multifiling remains projection-only. Mutation semantics are deliberately out
|
||||||
of scope for this read-side contract.
|
of scope for this read-side contract.
|
||||||
@@ -79,12 +81,17 @@ python3 -m pytest \
|
|||||||
Result: 21 passed, 16 skipped in 5.19s
|
Result: 21 passed, 16 skipped in 5.19s
|
||||||
```
|
```
|
||||||
|
|
||||||
Browser Binding API verification with optional service extras:
|
Focused CMIS verification with optional service extras:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
.venv/bin/python -m pytest tests/cmis/test_cmis_browser_binding_api.py -q
|
.venv/bin/python -m pytest \
|
||||||
|
tests/cmis/test_cmis_browser_binding_api.py \
|
||||||
|
tests/cmis/test_cmis_runtime_browser_binding.py \
|
||||||
|
tests/cmis/test_cmis_compliance_flags.py \
|
||||||
|
tests/cmis/test_cmis_contract_examples.py \
|
||||||
|
-q
|
||||||
|
|
||||||
Result: 16 passed in 39.09s
|
Result: 37 passed in 44.72s
|
||||||
```
|
```
|
||||||
|
|
||||||
The default system-Python run skips Browser Binding API tests when FastAPI/HTTPX
|
The default system-Python run skips Browser Binding API tests when FastAPI/HTTPX
|
||||||
@@ -92,3 +99,16 @@ are unavailable and skips capacity probes unless `KONTEXTUAL_RUN_CAPACITY=1` is
|
|||||||
set. The capacity probe exercises 400 documents and 250 relationships over
|
set. The capacity probe exercises 400 documents and 250 relationships over
|
||||||
query and target-filter paths, while relying on the shared performance history
|
query and target-filter paths, while relying on the shared performance history
|
||||||
monitor for drift tracking.
|
monitor for drift tracking.
|
||||||
|
|
||||||
|
OpenCMIS selected-baseline verification:
|
||||||
|
|
||||||
|
```text
|
||||||
|
run-20260514T003705Z
|
||||||
|
cmis-browser-baseline: completed
|
||||||
|
Guide Board summary: 2 pass, 1 warning
|
||||||
|
Policy: 0 unexpected findings
|
||||||
|
```
|
||||||
|
|
||||||
|
The only remaining OpenCMIS warning is local HTTP transport on the loopback
|
||||||
|
harness. `object-content` passes after child projections were made
|
||||||
|
deterministically ordered by `cmis:name`.
|
||||||
|
|||||||
@@ -30,8 +30,8 @@ Out of scope for `0.1.0`:
|
|||||||
| Area | Gate | Current state |
|
| Area | Gate | Current state |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| CMIS read-side contract | Query, navigation, relationship, ACL, and change-token contracts are release-stable or explicitly waived. | `KONT-WP-0016` implemented locally; full release verification still required. |
|
| CMIS read-side contract | Query, navigation, relationship, ACL, and change-token contracts are release-stable or explicitly waived. | `KONT-WP-0016` implemented locally; full release verification still required. |
|
||||||
| Tests | Full suite passes in the project venv. | `.venv/bin/python -m pytest -q` passed: `166 passed`, `15 skipped`; advisory performance drift warnings recorded. |
|
| Tests | Full suite passes in the project venv. | `.venv/bin/python -m pytest -q` passed: `166 passed`, `15 skipped` in `73.05s`; advisory performance drift warnings recorded. |
|
||||||
| CMIS evidence | OpenCMIS selected baseline completes with no unexpected findings. | `run-20260513T223537Z` completed; only local HTTP warning remains. |
|
| CMIS evidence | OpenCMIS selected baseline completes with no unexpected findings. | `run-20260514T003705Z` completed; object/content passed; only local HTTP warning remains. |
|
||||||
| Transport | Released CMIS access points are served behind HTTPS. | Required deployment gate; local loopback warning is accepted only for harness runs. |
|
| Transport | Released CMIS access points are served behind HTTPS. | Required deployment gate; local loopback warning is accepted only for harness runs. |
|
||||||
| Capability honesty | Scorecard, unsupported catalog, and examples match behavior. | Updated for `appendContentStream` and WP-0016 read-side contract; final doc review required. |
|
| Capability honesty | Scorecard, unsupported catalog, and examples match behavior. | Updated for `appendContentStream` and WP-0016 read-side contract; final doc review required. |
|
||||||
| Packaging | Version, dependencies, optional extras, and install smoke are checked. | `pyproject.toml` is already `0.1.0`; build/install smoke still required. |
|
| Packaging | Version, dependencies, optional extras, and install smoke are checked. | `pyproject.toml` is already `0.1.0`; build/install smoke still required. |
|
||||||
|
|||||||
@@ -1801,7 +1801,7 @@ class ServiceRuntime:
|
|||||||
for path in sorted(child_folder_paths)
|
for path in sorted(child_folder_paths)
|
||||||
if path not in workspace_folders
|
if path not in workspace_folders
|
||||||
]
|
]
|
||||||
return sorted(workspace_children + projection_children, key=lambda item: item["path"])
|
return sorted(workspace_children + projection_children, key=_cmis_child_order_key)
|
||||||
children: list[dict[str, Any]] = []
|
children: list[dict[str, Any]] = []
|
||||||
folder_path = _normalize_cmis_path(folder_path)
|
folder_path = _normalize_cmis_path(folder_path)
|
||||||
child_folder_paths: set[str] = set()
|
child_folder_paths: set[str] = set()
|
||||||
@@ -1833,7 +1833,7 @@ class ServiceRuntime:
|
|||||||
for path in sorted(child_folder_paths)
|
for path in sorted(child_folder_paths)
|
||||||
if path not in workspace_folders
|
if path not in workspace_folders
|
||||||
]
|
]
|
||||||
return projection_children + children
|
return sorted(projection_children + children, key=_cmis_child_order_key)
|
||||||
|
|
||||||
def _cmis_workspace_folder_map(self, access_point_id: str) -> dict[str, CMISWorkspaceFolder]:
|
def _cmis_workspace_folder_map(self, access_point_id: str) -> dict[str, CMISWorkspaceFolder]:
|
||||||
return self.cmis_workspace_folders.setdefault(access_point_id, {})
|
return self.cmis_workspace_folders.setdefault(access_point_id, {})
|
||||||
@@ -5050,6 +5050,12 @@ def _path_contains(parent_path: str, candidate_path: str) -> bool:
|
|||||||
return candidate == parent or candidate.startswith(parent.rstrip("/") + "/")
|
return candidate == parent or candidate.startswith(parent.rstrip("/") + "/")
|
||||||
|
|
||||||
|
|
||||||
|
def _cmis_child_order_key(item: dict[str, Any]) -> tuple[str, str]:
|
||||||
|
properties = dict(item.get("properties", {}))
|
||||||
|
name = properties.get("cmis:name") or item.get("name") or _path_name(str(item.get("path") or ""))
|
||||||
|
return (str(name).casefold(), str(item.get("object_id") or ""))
|
||||||
|
|
||||||
|
|
||||||
def _parse_multipart_form(content_type: str, body: bytes) -> tuple[dict[str, list[Any]], dict[str, dict[str, Any]]]:
|
def _parse_multipart_form(content_type: str, body: bytes) -> tuple[dict[str, list[Any]], dict[str, dict[str, Any]]]:
|
||||||
message = BytesParser(policy=policy.default).parsebytes(
|
message = BytesParser(policy=policy.default).parsebytes(
|
||||||
b"Content-Type: " + content_type.encode("utf-8") + b"\r\nMIME-Version: 1.0\r\n\r\n" + body
|
b"Content-Type: " + content_type.encode("utf-8") + b"\r\nMIME-Version: 1.0\r\n\r\n" + body
|
||||||
|
|||||||
@@ -183,10 +183,12 @@ def test_cmis_readonly_children_object_content_query_relationships_and_changes(c
|
|||||||
|
|
||||||
root_ids = {item["object_id"] for item in root_children["objects"]}
|
root_ids = {item["object_id"] for item in root_children["objects"]}
|
||||||
child_ids = {item["object_id"] for item in children["objects"]}
|
child_ids = {item["object_id"] for item in children["objects"]}
|
||||||
|
child_names = [item["name"] for item in children["objects"]]
|
||||||
assert "cmis:folder:assets" in root_ids
|
assert "cmis:folder:assets" in root_ids
|
||||||
assert "cmis:asset:asset-source" in child_ids
|
assert "cmis:asset:asset-source" in child_ids
|
||||||
assert "cmis:asset:asset-public" in child_ids
|
assert "cmis:asset:asset-public" in child_ids
|
||||||
assert "cmis:asset:asset-confidential" not in child_ids
|
assert "cmis:asset:asset-confidential" not in child_ids
|
||||||
|
assert child_names == sorted(child_names, key=str.casefold)
|
||||||
assert object_response["properties"]["kontextual:assetId"] == "asset-source"
|
assert object_response["properties"]["kontextual:assetId"] == "asset-source"
|
||||||
assert "get_content_stream" in object_response["allowable_actions"]
|
assert "get_content_stream" in object_response["allowable_actions"]
|
||||||
assert content["mime_type"] == "text/markdown"
|
assert content["mime_type"] == "text/markdown"
|
||||||
|
|||||||
@@ -96,9 +96,11 @@ def test_runtime_cmis_browser_repository_types_children_and_object(cmis_runtime)
|
|||||||
assert repository["unsupported_features"]["get_descendants"]["status"] == "unsupported"
|
assert repository["unsupported_features"]["get_descendants"]["status"] == "unsupported"
|
||||||
assert {item["base_type_id"] for item in types["items"]} >= {"cmis:document", "cmis:folder"}
|
assert {item["base_type_id"] for item in types["items"]} >= {"cmis:document", "cmis:folder"}
|
||||||
root_paths = {item["path"] for item in children["objects"]}
|
root_paths = {item["path"] for item in children["objects"]}
|
||||||
|
child_names = [item["name"] for item in topic_children["objects"]]
|
||||||
topic_object_ids = {item["object_id"] for item in topic_children["objects"]}
|
topic_object_ids = {item["object_id"] for item in topic_children["objects"]}
|
||||||
parent_paths = {item["path"] for item in parents["parents"]}
|
parent_paths = {item["path"] for item in parents["parents"]}
|
||||||
assert "/topics" in root_paths
|
assert "/topics" in root_paths
|
||||||
|
assert child_names == sorted(child_names, key=str.casefold)
|
||||||
assert "cmis:asset:asset-runtime-source" in topic_object_ids
|
assert "cmis:asset:asset-runtime-source" in topic_object_ids
|
||||||
assert "cmis:asset:asset-runtime-confidential" not in topic_object_ids
|
assert "cmis:asset:asset-runtime-confidential" not in topic_object_ids
|
||||||
assert {"/topics/cmis", "/topics/integration"} <= parent_paths
|
assert {"/topics/cmis", "/topics/integration"} <= parent_paths
|
||||||
|
|||||||
@@ -265,16 +265,19 @@ Result: 21 passed, 16 skipped in 5.19s
|
|||||||
|
|
||||||
The skipped tests require optional FastAPI/HTTPX test extras in this local
|
The skipped tests require optional FastAPI/HTTPX test extras in this local
|
||||||
environment. They remain part of the release verification gate when the service
|
environment. They remain part of the release verification gate when the service
|
||||||
extras are installed. No new OpenCMIS run was required for the selected
|
extras are installed.
|
||||||
object/content baseline because this workplan changes read-side query,
|
|
||||||
relationship, ACL, and diagnostics contracts outside that selected baseline.
|
|
||||||
|
|
||||||
Browser Binding API verification with service extras:
|
Focused CMIS verification with service extras:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
.venv/bin/python -m pytest tests/cmis/test_cmis_browser_binding_api.py -q
|
.venv/bin/python -m pytest \
|
||||||
|
tests/cmis/test_cmis_browser_binding_api.py \
|
||||||
|
tests/cmis/test_cmis_runtime_browser_binding.py \
|
||||||
|
tests/cmis/test_cmis_compliance_flags.py \
|
||||||
|
tests/cmis/test_cmis_contract_examples.py \
|
||||||
|
-q
|
||||||
|
|
||||||
Result: 16 passed in 39.09s
|
Result: 37 passed in 44.72s
|
||||||
```
|
```
|
||||||
|
|
||||||
Full suite verification with service extras:
|
Full suite verification with service extras:
|
||||||
@@ -282,13 +285,30 @@ Full suite verification with service extras:
|
|||||||
```text
|
```text
|
||||||
.venv/bin/python -m pytest -q
|
.venv/bin/python -m pytest -q
|
||||||
|
|
||||||
Result: 166 passed, 15 skipped in 55.94s
|
Result: 166 passed, 15 skipped in 73.05s
|
||||||
```
|
```
|
||||||
|
|
||||||
The run emitted advisory performance-drift warnings for several API tests. They
|
The run emitted advisory performance-drift warnings for several API tests. They
|
||||||
do not indicate functional failures and should be watched through the existing
|
do not indicate functional failures and should be watched through the existing
|
||||||
compact performance-history monitor.
|
compact performance-history monitor.
|
||||||
|
|
||||||
|
OpenCMIS release-readiness assessment:
|
||||||
|
|
||||||
|
```text
|
||||||
|
run-20260514T003705Z
|
||||||
|
Assessment: cmis-browser-baseline
|
||||||
|
Result: completed
|
||||||
|
Guide Board summary: 2 pass, 1 warning
|
||||||
|
Policy: 0 unexpected findings, 0 applied waivers
|
||||||
|
```
|
||||||
|
|
||||||
|
The first post-WP-0016 assessment run exposed an avoidable `cmis:name` child
|
||||||
|
ordering warning after `capabilityOrderBy=common` was advertised. CMIS children
|
||||||
|
are now sorted deterministically by `cmis:name`; the final persisted run has
|
||||||
|
`object-content` passing and only the known local HTTP transport warning
|
||||||
|
remaining. Evidence is persisted in
|
||||||
|
`docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-14T003705Z.md`.
|
||||||
|
|
||||||
## Release Advice
|
## Release Advice
|
||||||
|
|
||||||
This workplan should run before `KONT-WP-0015` unless the first release is
|
This workplan should run before `KONT-WP-0015` unless the first release is
|
||||||
|
|||||||
Reference in New Issue
Block a user