CMIS Capability Scorecard

docs/cmis-1-1-capability-scorecard.md

@@ -0,0 +1,142 @@
+# CMIS 1.1 Capability Scorecard
+
+Date: 2026-05-07
+
+Status: baseline scorecard for the current Browser Binding subset.
+
+## Purpose
+
+Estimate how deep the current `kontextual-engine` CMIS 1.1 surface is when
+compared with a mature, market-grade content services implementation.
+
+This scorecard is not a certification claim and not a judgment of the engine's
+native architecture. It measures only CMIS compatibility depth: how much a
+generic CMIS client can expect to use without knowing Kontextual-specific
+semantics.
+
+## Benchmark
+
+Primary CMIS-depth benchmark: **Hyland Alfresco Content Services**.
+
+Rationale:
+
+- Hyland/Alfresco documentation states that Alfresco Content Services fully
+  implements CMIS 1.0 and CMIS 1.1.
+- The same documentation describes CMIS Browser, AtomPub, and Web Services
+  bindings as supported CMIS client entry points.
+- Hyland is a recognized content services/document management vendor; Gartner's
+  2024 Document Management Magic Quadrant vendor list includes Hyland,
+  Laserfiche, M-Files, Microsoft, OpenText, and others.
+
+Secondary market comparators for product context: OpenText, Hyland OnBase,
+M-Files, and Laserfiche. These are useful commercial reference points, but
+Hyland Alfresco is the cleanest CMIS-specific benchmark because its public docs
+make a direct full-CMIS implementation claim.
+
+Sources:
+
+- OASIS CMIS 1.1 specification:
+  <https://docs.oasis-open.org/cmis/CMIS/v1.1/os/CMIS-v1.1-os.html>
+- Hyland Alfresco Content Services 25.1 CMIS API docs:
+  <https://support.hyland.com/r/Alfresco/Alfresco-Content-Services/25.1/Alfresco-Content-Services/Develop/Reference/CMIS-API>
+- Gartner 2024 Document Management Magic Quadrant vendor list:
+  <https://www.gartner.com/en/documents/6030235>
+
+## Scoring Rubric
+
+| Score band | Meaning |
+| --- | --- |
+| 0-10% | Absent or intentionally unsupported. |
+| 15-30% | Projection, metadata, or diagnostics exist, but CMIS clients cannot rely on the full service. |
+| 35-55% | Useful constrained subset with explicit unsupported flags. |
+| 60-75% | Production-usable subset for controlled clients, still missing notable optional or advanced behavior. |
+| 80-95% | Mature implementation with broad client compatibility but not necessarily certified. |
+| 100% | Benchmark-grade full CMIS 1.1 behavior for the capability area. |
+
+Overall score uses weighted capability areas. The weights estimate practical
+CMIS interoperability importance rather than engine-internal importance.
+
+## Overall Estimate
+
+| Metric | Score |
+| --- | ---: |
+| Weighted CMIS 1.1 depth vs Hyland Alfresco benchmark | 42% |
+| Controlled-client Browser Binding usefulness | 58% |
+| Broad commodity CMIS client compatibility | 35% |
+
+Interpretation: the current CMIS layer is a credible Browser Binding subset for
+known clients and profile-specific integrations. It is not yet a broad ECM/CMIS
+replacement surface.
+
+## Capability Scorecard
+
+| CMIS capability area | Weight | Current depth | Most worthy contender | Gap basis behind the percentage |
+| --- | ---: | ---: | --- | --- |
+| Repository service and repository info | 5 | 75% | Hyland Alfresco ACS | - Repository info and conservative capability flags exist.<br>- Unsupported feature catalog exists.<br>- Missing exact service-document parity and external TCK evidence. |
+| Type definitions | 6 | 45% | Hyland Alfresco ACS | - Base types and content stream properties exist.<br>- No mutable types or custom schema/type management.<br>- No broad property definition model beyond current projected fields. |
+| Navigation service | 8 | 40% | Hyland Alfresco ACS | - Root and folder-scoped children exist.<br>- Projection-only parents exist.<br>- Missing `getDescendants`, `getFolderTree`, object-by-path parity, and real filing mutations. |
+| Object read service | 10 | 70% | Hyland Alfresco ACS | - Object envelopes, properties, content descriptors, ACL projection, relationships, and allowable actions exist.<br>- Missing selector/property-filter fidelity and full Browser Binding response parity.<br>- Deleted/hidden objects are now correctly not exposed. |
+| Object write service | 8 | 35% | Hyland Alfresco ACS | - `createDocument`, custom metadata updates, content stream set, and delete-request lifecycle exist.<br>- No createFolder, moveObject, standard `cmis:*` property mutation, or physical delete semantics.<br>- Delete is intentionally governed, not raw repository removal. |
+| Content stream read/write | 8 | 65% | Hyland Alfresco ACS | - Byte streaming and deduplicating `setContentStream` exist.<br>- Digest verification and governed access exist.<br>- Missing append/delete stream, multipart Browser Binding parity, range handling, and client-tested large stream workflows. |
+| Versioning service | 8 | 25% | Hyland Alfresco ACS | - Version properties can be projected from engine versions.<br>- No checkout/checkin/cancelCheckout/PWC services.<br>- No version history route or all-versions query behavior. |
+| Discovery/query | 8 | 25% | Hyland Alfresco ACS | - Narrow document select subset exists.<br>- Unsupported joins/order-by return diagnostics.<br>- Missing CMIS SQL predicates, type joins, full-text, ordering, and rich projection rules. |
+| Relationships | 5 | 60% | Hyland Alfresco ACS | - Relationship object projection and source filtering exist.<br>- Visibility gates prevent protected relationship leakage.<br>- Missing full relationship service filters, relationship creation through CMIS, and type hierarchy maturity. |
+| ACL service | 6 | 35% | Hyland Alfresco ACS | - Discover-only ACL projection exists.<br>- `applyACL` is blocked as not implemented.<br>- Missing inherited/direct ACL fidelity, propagation, ACL mutation, and repository principal model. |
+| Policy service | 3 | 10% | Hyland Alfresco ACS | - Native policy decisions govern exposure.<br>- No CMIS policy objects, `applyPolicy`, `removePolicy`, or `getAppliedPolicies` service surface.<br>- Explicitly unsupported. |
+| Change log | 5 | 55% | Hyland Alfresco ACS | - Audit-backed object-id change entries and paging exist.<br>- Missing full change token durability semantics and richer change event typing.<br>- Not yet proven against external CMIS clients. |
+| Multi-filing and unfiling | 4 | 25% | Hyland Alfresco ACS | - Projection-only parent maps exist and are useful for navigation.<br>- Standard CMIS `capabilityMultifiling` is correctly false.<br>- No add/remove filing mutations or canonical folder membership model. |
+| Renditions | 3 | 15% | Hyland Alfresco ACS | - Native representations could become rendition candidates later.<br>- CMIS rendition capability is currently `none`.<br>- No rendition taxonomy or rendition stream routes. |
+| Retention and hold | 2 | 5% | OpenText / Hyland governance stacks | - Native governance metadata can represent intent later.<br>- No CMIS retention/hold model or mutation services.<br>- Explicitly unsupported. |
+| Bulk update | 2 | 5% | Hyland Alfresco ACS | - Native batch/error envelopes exist elsewhere in the engine.<br>- No CMIS `bulkUpdateProperties` behavior.<br>- Explicitly unsupported. |
+| Browser Binding protocol fidelity | 7 | 45% | Hyland Alfresco ACS | - Browser-style routes and JSON envelopes exist.<br>- FastAPI route shapes are pragmatic, not complete CMIS Browser Binding selector/action parity.<br>- Route-level tests skip without optional service dependencies. |
+| AtomPub binding | 2 | 0% | Hyland Alfresco ACS | - No AtomPub/XML service document or feeds.<br>- Intentionally deferred until monetized need. |
+| Web Services binding | 2 | 0% | Hyland Alfresco ACS | - No SOAP/WSDL stack.<br>- Intentionally deferred until monetized need. |
+| External conformance evidence | 3 | 20% | OpenCMIS TCK against Alfresco-like server behavior | - Internal fixtures and optional TCK mapping exist.<br>- No recorded OpenCMIS TCK execution against a running access point yet.<br>- No third-party client compatibility matrix yet. |
+
+Weighted result from this table: **42%**.
+
+## Most Important Gaps
+
+1. **External conformance run**
+   - Run selected OpenCMIS TCK groups against `compat-tck`.
+   - Capture failures by capability group.
+   - Turn "estimated" scores into evidence-backed scores.
+
+2. **Browser Binding fidelity**
+   - Align route/action/selector shapes more closely with CMIS Browser Binding.
+   - Add non-skipped FastAPI route tests in CI with service extras installed.
+   - Add client smoke tests with Apache Chemistry/OpenCMIS where feasible.
+
+3. **Query depth**
+   - Add a real CMIS SQL subset parser instead of a two-query allowlist.
+   - Support basic `WHERE`, equality predicates, paging, ordering where claimed,
+     and diagnostics for everything outside the subset.
+
+4. **Navigation depth**
+   - Decide whether `getDescendants` and `getFolderTree` are worth implementing
+     over projection-only folders.
+   - Keep mutating filing unsupported unless a client explicitly needs it.
+
+5. **Versioning depth**
+   - Keep PWC/checkin/checkout unsupported for now.
+   - Add version-history read support if document-management integrations begin
+     depending on it.
+
+6. **Renditions**
+   - Map selected derived representations to CMIS renditions only after we have
+     stable representation taxonomy and real preview/thumbnail use cases.
+
+## Product Positioning Takeaway
+
+Against a mature CMIS implementation such as Hyland Alfresco ACS, Kontextual is
+not trying to win by being a full ECM clone. Its current strength is a governed,
+profiled, source-grounded knowledge runtime with enough CMIS Browser Binding
+surface for controlled interoperability.
+
+The right strategic posture is therefore:
+
+- advertise a conservative CMIS 1.1 Browser Binding subset,
+- keep unsupported capability flags honest,
+- use the scorecard to decide which gaps are worth closing for real clients,
+- avoid AtomPub/Web Services and full ECM semantics unless revenue or a named
+  integration justifies the complexity.