Log analysis functionality for self-assessment

docs/LOCAL-RUNBOOK.md

@@ -193,12 +193,35 @@ PYTHONPATH=src python3 scripts/cmis_scorecard.py \
   --run-dir /tmp/open-cmis-tck-live
 ```
 
+Generate the log review after the scorecard:
+
+```sh
+PYTHONPATH=src python3 scripts/opencmis_log_review.py \
+  --run-dir /tmp/open-cmis-tck-live
+```
+
+The review classifies warnings, hard errors, stderr, skipped cases, and closed
+warnings when a previous run is supplied. The default warning policy accepts the
+OpenCMIS HTTP transport warning only for local/test loopback profiles.
+
+For important product assessments, archive the run before `/tmp` cleanup:
+
+```sh
+PYTHONPATH=src python3 scripts/archive_assessment_run.py \
+  --run-dir /tmp/open-cmis-tck-live \
+  --archive-root .local/runs/archive
+```
+
+The archive writes `.local/runs/archive/<target>/<run-id>/archive-manifest.json`
+with SHA-256 hashes for every copied file.
+
 Review the real TCK evidence before expanding the scope:
 
 ```text
 /tmp/open-cmis-tck-live/normalized/evidence.json
 /tmp/open-cmis-tck-live/artifacts/open-cmis-tck/tck/<check-group>/console-runner-stdout.txt
 /tmp/open-cmis-tck-live/artifacts/open-cmis-tck/tck/<check-group>/normalized-runner-result.json
+/tmp/open-cmis-tck-live/reports/opencmis-log-review.md
 ```
 
 The normalizer preserves native OpenCMIS statuses (`OK`, `WARNING`, `FAILURE`,

docs/LOG-REVIEW.md

@@ -0,0 +1,132 @@
+# OpenCMIS Log Review
+
+Status: draft
+Created: 2026-05-14
+
+## Purpose
+
+The log review command turns a guide-board OpenCMIS run directory into a compact
+fix-or-acceptance report. It reads normalized OpenCMIS cases, findings,
+stderr artifacts, optional server logs, and the warning policy profile.
+
+It answers four operator questions:
+
+- Did any hard OpenCMIS case fail?
+- Did the runner or server write unexpected stderr/log errors?
+- Are warnings accepted local-test conditions or release blockers?
+- Do skipped cases match the target's advertised capability boundary?
+
+## Command
+
+Generate a review after a run:
+
+```sh
+cd /home/worsch/open-cmis-tck
+PYTHONPATH=src python3 scripts/opencmis_log_review.py \
+  --run-dir /tmp/open-cmis-tck-live
+```
+
+The command writes:
+
+```text
+/tmp/open-cmis-tck-live/reports/opencmis-log-review.json
+/tmp/open-cmis-tck-live/reports/opencmis-log-review.md
+```
+
+Compare with a previous run to surface closed and new warnings:
+
+```sh
+PYTHONPATH=src python3 scripts/opencmis_log_review.py \
+  --run-dir /tmp/kontextual-cmis-release-20260514-toolchain \
+  --previous-run-dir /tmp/open-cmis-tck-kontextual-20260513T230205Z
+```
+
+Include known server logs when available:
+
+```sh
+PYTHONPATH=src python3 scripts/opencmis_log_review.py \
+  --run-dir .local/runs/opencmis-inmemory-pilot \
+  --server-log .local/opencmis-inmemory/logs \
+  --server-log .local/opencmis-inmemory/containers/apache-tomcat-9.0.117/logs
+```
+
+Server-log matches are reported as review context. They do not change the
+overall review status by themselves because external log directories often
+contain historical startup attempts from before the assessed run.
+
+## Warning Policy
+
+The default policy is:
+
+```text
+profiles/expectations/opencmis-warning-policy.json
+```
+
+It currently classifies:
+
+- `HTTPS is not used...` as acceptable only for explicit local/test loopback
+  HTTP endpoints. The same warning becomes a deployment transport blocker for
+  non-loopback or production-like targets.
+- `Thin client URI is not set!` as an accepted limitation only for the
+  `opencmis-inmemory-local` self-test target.
+
+Policies intentionally live outside narrative evidence. A release run should
+show whether each warning is accepted by policy, not rely on a human remembering
+the rule.
+
+## Skip Interpretation
+
+Skipped OpenCMIS cases are grouped by capability boundary. A skip is treated as
+expected when the corresponding capability is not advertised by the target
+profile. For example:
+
+- non-creatable `cmis:relationship` -> requires `cmis.relationships`
+- non-creatable `cmis:policy` -> requires `cmis.policy-mutation`
+- non-creatable `cmis:item` -> requires `cmis.item-services`
+- document sub-type mutation -> requires `cmis.type-mutability`
+- folder-name mutation -> requires `cmis.folder-name-mutation`
+
+If a target advertises the relevant capability and the TCK still skips the case,
+the review status becomes `review_required`.
+
+## Durable Archive
+
+Do not leave important product assessment evidence only under `/tmp`. Archive a
+run after scorecard/log-review generation:
+
+```sh
+cd /home/worsch/open-cmis-tck
+PYTHONPATH=src python3 scripts/archive_assessment_run.py \
+  --run-dir /tmp/kontextual-cmis-release-20260514-toolchain \
+  --archive-root .local/runs/archive
+```
+
+The archive command copies the run directory and writes:
+
+```text
+.local/runs/archive/<target>/<run-id>/archive-manifest.json
+```
+
+The manifest records the source path, archive path, run metadata, file count,
+total bytes, and SHA-256 for every copied file.
+
+## Next Coverage Frontier
+
+Keep the default baseline at `repository-type` plus `object-content` until
+warning policy and durable evidence are routine. The next recommended maturity
+slice is navigation/read-path depth, because it sits closest to current
+object/content behavior and reveals path, parent, children, and filing semantics
+without immediately requiring versioning or full query depth.
+
+Candidate order:
+
+1. Navigation/read-path checks with unsupported filing mutations clearly scoped.
+2. Metadata query checks for the advertised `metadataonly` query posture.
+3. ACL/policy discovery depth before ACL/policy mutation.
+4. Versioning/PWC and change-log depth only after the product deliberately
+   advertises those capabilities.
+
+## Boundary
+
+This log review supports preparation evidence and operational readiness. It
+does not issue CMIS certification or audit assurance.

docs/SERVICE-AND-RETENTION.md

@@ -39,6 +39,13 @@ CMIS runs use the guide-board run directory contract. Each run writes:
 - `reports/assessment-package.json`
 - `reports/report.md`
 
+Extension-side post-processing can also add:
+
+- `reports/cmis-maturity-scorecard.json`
+- `reports/cmis-maturity-scorecard.md`
+- `reports/opencmis-log-review.json`
+- `reports/opencmis-log-review.md`
+
 The sample assessment profile keeps summaries for 365 days and raw artifacts for
 30 days:
 
@@ -54,3 +61,23 @@ The sample assessment profile keeps summaries for 365 days and raw artifacts for
 Compact `retention-summary.json` files are suitable for guide-board trend
 summaries and downstream CMIS capability scorecards without retaining unbounded
 raw TCK logs.
+
+## Durable Local Archive
+
+For product-facing assessment evidence, do not rely on `/tmp` as the only copy.
+Archive completed runs after scorecard and log-review generation:
+
+```sh
+cd /home/worsch/open-cmis-tck
+PYTHONPATH=src python3 scripts/archive_assessment_run.py \
+  --run-dir /tmp/open-cmis-tck-live \
+  --archive-root .local/runs/archive
+```
+
+The archive command copies the full run directory and writes
+`archive-manifest.json` with SHA-256 hashes, file sizes, source path, archive
+path, run ID, target profile reference, and assessment profile reference.
+
+The default local archive root remains under `.local/`, so it is not committed.
+Move selected archive packages into a controlled evidence store when the run is
+used for release or external audit preparation.