open-cmis-tck/docs/LOG-REVIEW.md

OpenCMIS Log Review

Status: draft Created: 2026-05-14

Purpose

The log review command turns a guide-board OpenCMIS run directory into a compact fix-or-acceptance report. It reads normalized OpenCMIS cases, findings, stderr artifacts, optional server logs, and the warning policy profile.

It answers four operator questions:

• Did any hard OpenCMIS case fail?
• Did the runner or server write unexpected stderr/log errors?
• Are warnings accepted local-test conditions or release blockers?
• Do skipped cases match the target's advertised capability boundary?

Command

Generate a review after a run:

cd /home/worsch/open-cmis-tck
PYTHONPATH=src python3 scripts/opencmis_log_review.py \
  --run-dir /tmp/open-cmis-tck-live

The command writes:

/tmp/open-cmis-tck-live/reports/opencmis-log-review.json
/tmp/open-cmis-tck-live/reports/opencmis-log-review.md

Compare with a previous run to surface closed and new warnings:

PYTHONPATH=src python3 scripts/opencmis_log_review.py \
  --run-dir /tmp/kontextual-cmis-release-20260514-toolchain \
  --previous-run-dir /tmp/open-cmis-tck-kontextual-20260513T230205Z

Include known server logs when available:

PYTHONPATH=src python3 scripts/opencmis_log_review.py \
  --run-dir .local/runs/opencmis-inmemory-pilot \
  --server-log .local/opencmis-inmemory/logs \
  --server-log .local/opencmis-inmemory/containers/apache-tomcat-9.0.117/logs

Server-log matches are reported as review context. They do not change the overall review status by themselves because external log directories often contain historical startup attempts from before the assessed run.

Warning Policy

The default policy is:

profiles/expectations/opencmis-warning-policy.json

It currently classifies:

• HTTPS is not used... as acceptable only for explicit local/test loopback HTTP endpoints. The same warning becomes a deployment transport blocker for non-loopback or production-like targets.
• Thin client URI is not set! as an accepted limitation only for the opencmis-inmemory-local self-test target.

Policies intentionally live outside narrative evidence. A release run should show whether each warning is accepted by policy, not rely on a human remembering the rule.

Skip Interpretation

Skipped OpenCMIS cases are grouped by capability boundary. A skip is treated as expected when the corresponding capability is not advertised by the target profile. For example:

• non-creatable cmis:relationship -> requires cmis.relationships
• non-creatable cmis:policy -> requires cmis.policy-mutation
• non-creatable cmis:item -> requires cmis.item-services
• document sub-type mutation -> requires cmis.type-mutability
• folder-name mutation -> requires cmis.folder-name-mutation

If a target advertises the relevant capability and the TCK still skips the case, the review status becomes review_required.

Durable Archive

Do not leave important product assessment evidence only under /tmp. Archive a run after scorecard/log-review generation:

cd /home/worsch/open-cmis-tck
PYTHONPATH=src python3 scripts/archive_assessment_run.py \
  --run-dir /tmp/kontextual-cmis-release-20260514-toolchain \
  --archive-root .local/runs/archive

The archive command copies the run directory and writes:

.local/runs/archive/<target>/<run-id>/archive-manifest.json

The manifest records the source path, archive path, run metadata, file count, total bytes, and SHA-256 for every copied file.

Next Coverage Frontier

Keep the default baseline at repository-type plus object-content until warning policy and durable evidence are routine. The next recommended maturity slice is navigation/read-path depth, because it sits closest to current object/content behavior and reveals path, parent, children, and filing semantics without immediately requiring versioning or full query depth.

Candidate order:

1. Navigation/read-path checks with unsupported filing mutations clearly scoped.
2. Metadata query checks for the advertised metadataonly query posture.
3. ACL/policy discovery depth before ACL/policy mutation.
4. Versioning/PWC and change-log depth only after the product deliberately advertises those capabilities.

Boundary

This log review supports preparation evidence and operational readiness. It does not issue CMIS certification or audit assurance.