Log analysis functionality for self-assessment

docs/LOG-REVIEW.md

@@ -0,0 +1,132 @@
+# OpenCMIS Log Review
+
+Status: draft
+Created: 2026-05-14
+
+## Purpose
+
+The log review command turns a guide-board OpenCMIS run directory into a compact
+fix-or-acceptance report. It reads normalized OpenCMIS cases, findings,
+stderr artifacts, optional server logs, and the warning policy profile.
+
+It answers four operator questions:
+
+- Did any hard OpenCMIS case fail?
+- Did the runner or server write unexpected stderr/log errors?
+- Are warnings accepted local-test conditions or release blockers?
+- Do skipped cases match the target's advertised capability boundary?
+
+## Command
+
+Generate a review after a run:
+
+```sh
+cd /home/worsch/open-cmis-tck
+PYTHONPATH=src python3 scripts/opencmis_log_review.py \
+  --run-dir /tmp/open-cmis-tck-live
+```
+
+The command writes:
+
+```text
+/tmp/open-cmis-tck-live/reports/opencmis-log-review.json
+/tmp/open-cmis-tck-live/reports/opencmis-log-review.md
+```
+
+Compare with a previous run to surface closed and new warnings:
+
+```sh
+PYTHONPATH=src python3 scripts/opencmis_log_review.py \
+  --run-dir /tmp/kontextual-cmis-release-20260514-toolchain \
+  --previous-run-dir /tmp/open-cmis-tck-kontextual-20260513T230205Z
+```
+
+Include known server logs when available:
+
+```sh
+PYTHONPATH=src python3 scripts/opencmis_log_review.py \
+  --run-dir .local/runs/opencmis-inmemory-pilot \
+  --server-log .local/opencmis-inmemory/logs \
+  --server-log .local/opencmis-inmemory/containers/apache-tomcat-9.0.117/logs
+```
+
+Server-log matches are reported as review context. They do not change the
+overall review status by themselves because external log directories often
+contain historical startup attempts from before the assessed run.
+
+## Warning Policy
+
+The default policy is:
+
+```text
+profiles/expectations/opencmis-warning-policy.json
+```
+
+It currently classifies:
+
+- `HTTPS is not used...` as acceptable only for explicit local/test loopback
+  HTTP endpoints. The same warning becomes a deployment transport blocker for
+  non-loopback or production-like targets.
+- `Thin client URI is not set!` as an accepted limitation only for the
+  `opencmis-inmemory-local` self-test target.
+
+Policies intentionally live outside narrative evidence. A release run should
+show whether each warning is accepted by policy, not rely on a human remembering
+the rule.
+
+## Skip Interpretation
+
+Skipped OpenCMIS cases are grouped by capability boundary. A skip is treated as
+expected when the corresponding capability is not advertised by the target
+profile. For example:
+
+- non-creatable `cmis:relationship` -> requires `cmis.relationships`
+- non-creatable `cmis:policy` -> requires `cmis.policy-mutation`
+- non-creatable `cmis:item` -> requires `cmis.item-services`
+- document sub-type mutation -> requires `cmis.type-mutability`
+- folder-name mutation -> requires `cmis.folder-name-mutation`
+
+If a target advertises the relevant capability and the TCK still skips the case,
+the review status becomes `review_required`.
+
+## Durable Archive
+
+Do not leave important product assessment evidence only under `/tmp`. Archive a
+run after scorecard/log-review generation:
+
+```sh
+cd /home/worsch/open-cmis-tck
+PYTHONPATH=src python3 scripts/archive_assessment_run.py \
+  --run-dir /tmp/kontextual-cmis-release-20260514-toolchain \
+  --archive-root .local/runs/archive
+```
+
+The archive command copies the run directory and writes:
+
+```text
+.local/runs/archive/<target>/<run-id>/archive-manifest.json
+```
+
+The manifest records the source path, archive path, run metadata, file count,
+total bytes, and SHA-256 for every copied file.
+
+## Next Coverage Frontier
+
+Keep the default baseline at `repository-type` plus `object-content` until
+warning policy and durable evidence are routine. The next recommended maturity
+slice is navigation/read-path depth, because it sits closest to current
+object/content behavior and reveals path, parent, children, and filing semantics
+without immediately requiring versioning or full query depth.
+
+Candidate order:
+
+1. Navigation/read-path checks with unsupported filing mutations clearly scoped.
+2. Metadata query checks for the advertised `metadataonly` query posture.
+3. ACL/policy discovery depth before ACL/policy mutation.
+4. Versioning/PWC and change-log depth only after the product deliberately
+   advertises those capabilities.
+
+## Boundary
+
+This log review supports preparation evidence and operational readiness. It
+does not issue CMIS certification or audit assurance.