open-cmis-tck/workplans/OPEN-CMIS-TCK-WP-0003-assessment-log-review-and-hardening.md

id, type, title, repo, extension, domain, status, owner, planning_priority, planning_order, created, updated, depends_on, state_hub_workstream_id

id	type	title	repo	extension	domain	status	owner	planning_priority	planning_order	created	updated	depends_on	state_hub_workstream_id
OPEN-CMIS-TCK-WP-0003	extension-workplan	Assessment Log Review And Hardening	open-cmis-tck	open-cmis-tck	markitect	completed	codex	high	4	2026-05-14	2026-05-14	OPEN-CMIS-TCK-WP-0002	5711ee2f-eaa9-428a-a4b2-e7383bfbf18a

OPEN-CMIS-TCK-WP-0003: Assessment Log Review And Hardening

Purpose

Use the first real kontextual-engine OpenCMIS TCK assessment runs to harden the open-cmis-tck guide-board extension around warning policy, durable evidence retention, and repeatable log review.

The latest kontextual-engine release-readiness run is healthy for the selected Browser Binding baseline: no hard TCK failures, no infrastructure errors, no unexpected findings, and empty stderr artifacts. The remaining work is mostly facility maturity: make the one current warning intentional, preserve raw evidence outside ephemeral /tmp paths, and give future assessments a compact "what should we fix next" report instead of relying on manual rg passes.

Evidence Reviewed

• Latest release-readiness evidence: /home/worsch/kontextual-engine/docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-13T223537Z.md
• Latest raw release-readiness run: /tmp/kontextual-cmis-release-20260514-toolchain
• Prior raw run before appendContentStream() support: /tmp/open-cmis-tck-kontextual-20260513T230205Z
• Earlier implementation evidence: /home/worsch/kontextual-engine/docs/cmis-opencmis-tck-implementation-evidence-2026-05-08T092113Z.md
• Local extension self-test run: /home/worsch/open-cmis-tck/.local/runs/opencmis-inmemory-pilot
• Local OpenCMIS in-memory server logs: /home/worsch/open-cmis-tck/.local/opencmis-inmemory/logs

Current Findings

1. The latest kontextual-engine run completed with Guide Board summary pass: 2, warning: 1, unexpected_findings: 0.
2. console-runner-stderr.txt and stderr.log are empty for both selected TCK groups in the latest run.
3. The remaining current warning is from OpenCMIS SecurityTest.java:67: HTTPS is not used. Credentials might be transferred as plain text!
4. The previous appendContentStream() warning in SetAndDeleteContentTest.java:200 is closed in the latest run.
5. The latest object/content run still has skipped cases for non-creatable relationship, policy, and item types, plus folder-name change-token subcases. These align with declared capability boundaries and are not errors, but they should remain visible as maturity scope.
6. The local OpenCMIS in-memory pilot has two repository/type warnings: loopback HTTP and Thin client URI is not set!. Tomcat and in-memory server logs did not show warning/error/exception lines in the scan.
7. Evidence retention is fragile: several useful raw runs live under /tmp, and at least one earlier /tmp run was already unavailable when later evidence was written.

Boundary

This workplan hardens the open-cmis-tck extension and its assessment operations. Product changes for kontextual-engine belong in that repository. This workplan may document product-facing follow-up candidates, but it should not modify the product repo directly.

D3.1 - Capture Current Log Triage Baseline

id: OPEN-CMIS-TCK-WP-0003-T001
status: done
priority: high
state_hub_task_id: "1a262cad-a945-4a93-a957-02f2fdb497f1"

Acceptance:

• Inspect the latest persisted evidence and raw run artifacts for kontextual-engine.
• Separate current findings from older findings that have already been closed.
• Inspect the local in-memory pilot logs so extension self-test warnings are not confused with product warnings.
• Record the baseline in this workplan.

Progress:

• Confirmed the latest raw release-readiness run has no fail, infrastructure_error, unexpected finding, stderr output, or exception trace.
• Confirmed the only current kontextual-engine TCK warning is local HTTP transport.
• Confirmed appendContentStream() was a warning in the prior raw run and is gone in the latest raw run.
• Confirmed the local in-memory pilot still reports loopback HTTP and missing thin-client URI warnings, while server logs are clean.

D3.2 - Durable Assessment Archive Path

id: OPEN-CMIS-TCK-WP-0003-T002
status: done
priority: high
state_hub_task_id: "1e31b306-f21e-4bac-8e69-56d586d6712e"

Acceptance:

• Provide a recommended non-ephemeral output layout for local product assessments, for example .local/runs/<target>/<run-id> or a configured workspace archive path.
• Add an operator command or documented copy/import step that preserves raw TCK stdout/stderr, normalized evidence, findings, mappings, run metadata, report, scorecard, and artifact manifest before /tmp cleanup can remove them.
• Preserve artifact hashes or package metadata so copied evidence remains auditable.
• Update the local runbook and service/retention notes with the durable path.

Progress:

• Added src/open_cmis_tck/archive.py and scripts/archive_assessment_run.py.
• The archive command copies a run into .local/runs/archive/<target>/<run-id> by default and writes archive-manifest.json with SHA-256 hashes, file sizes, source path, archive path, run ID, target profile reference, and assessment profile reference.
• Updated README, local runbook, and service/retention docs with the archive command.
• Archived the latest kontextual release-readiness run to .local/runs/archive/kontextual-cmis-compat/run-20260513T223537Z.

D3.3 - Warning Policy And HTTPS Deployment Gate

id: OPEN-CMIS-TCK-WP-0003-T003
status: done
priority: high
state_hub_task_id: "58d2cf64-0db6-4f9a-a8d5-df9ef870557b"

Acceptance:

• Define how warning policy distinguishes local loopback test topology from a deployment or release gate.
• Treat the OpenCMIS HTTP warning as acceptable only for explicit local loopback profiles or documented local waivers.
• Make non-loopback or release-target HTTP warnings visible as deployment blockers, even when the TCK group return code is 0.
• Record warning policy in a profile, expectation, or waiver file rather than burying it in narrative evidence.

Progress:

• Added profiles/expectations/opencmis-warning-policy.json.
• Classified the OpenCMIS HTTP warning as accepted only for local/test loopback HTTP endpoints.
• Non-loopback or production-like HTTP warnings are now classified as deployment_transport_blocker by the log-review command.

D3.4 - OpenCMIS In-Memory Pilot Warning Cleanup

id: OPEN-CMIS-TCK-WP-0003-T004
status: done
priority: medium
state_hub_task_id: "12331abc-c28f-4140-9a04-a70eb761bccf"

Acceptance:

• Investigate whether the local OpenCMIS in-memory server can expose a thinClientURI through configuration.
• If the upstream in-memory server cannot be configured cleanly, mark the warning as an expected self-test limitation with a precise source location and explanation.
• Keep the in-memory pilot useful as an extension smoke test without making its target-specific warnings look like guide-board defects.
• Document the expected warning posture in the local runbook.

Progress:

• Added an explicit policy entry for the opencmis-inmemory-local Thin client URI is not set! warning.
• The in-memory pilot review now classifies loopback HTTP and missing thin-client URI as accepted local self-test warnings.
• Optional external server-log findings are reported as context without changing the run status by themselves, because those log directories may include historical startup attempts outside the assessed run.

D3.5 - Automated Log Review Report

id: OPEN-CMIS-TCK-WP-0003-T005
status: done
priority: high
state_hub_task_id: "e445f909-678b-4236-a025-d5913e5473ed"

Acceptance:

• Add a command that scans a guide-board run directory for OpenCMIS stdout, stderr, normalized results, findings, and known server logs.
• Generate reports/opencmis-log-review.json and reports/opencmis-log-review.md.
• Highlight hard errors, non-empty stderr, new warnings, known accepted warnings, skipped cases, unexpected findings, and closed-warning comparisons when a previous run is supplied.
• Add regression tests with sanitized fixtures for the current HTTP warning, the now-closed appendContentStream() warning, empty stderr, and skipped capability-boundary cases.

Progress:

• Added src/open_cmis_tck/log_review.py and scripts/opencmis_log_review.py.
• The command writes reports/opencmis-log-review.json and reports/opencmis-log-review.md.
• Verified it against /tmp/kontextual-cmis-release-20260514-toolchain with /tmp/open-cmis-tck-kontextual-20260513T230205Z as the previous run.
• Added regression coverage for accepted HTTP warnings, non-loopback deployment blockers, closed append warnings, stderr handling, and skipped capability boundaries.

D3.6 - Skip And Capability Boundary Interpretation

id: OPEN-CMIS-TCK-WP-0003-T006
status: done
priority: medium
state_hub_task_id: "6df3e8c5-c2d2-4eb3-973e-76644ef7ee8f"

Acceptance:

• Group skipped OpenCMIS cases by declared repository capability or type creatability boundary.
• Distinguish "expected because the capability is not advertised" from "skipped because the target could not exercise an advertised capability."
• Keep skipped cases visible in reports and maturity scorecards without treating them as failures when they match the advertised capability profile.
• Add coverage for relationship, policy, item, type-subtype, and folder-name change-token skips seen in the latest raw run.

Progress:

• Added skip-boundary classification in the log-review report.
• Current expected skip rules cover relationship, policy, item, document subtype, and folder-name mutation cases.
• If the target advertises the required capability and OpenCMIS still skips the case, the review becomes review_required.

D3.7 - Next Coverage Frontier

id: OPEN-CMIS-TCK-WP-0003-T007
status: done
priority: medium
state_hub_task_id: "f793e1b8-a2b1-4f9b-9972-b6b18d1ba56a"

Acceptance:

• Identify which additional OpenCMIS TCK groups are realistic after the current repository/type and object/content baseline.
• For each candidate group, record target preconditions, likely product capability requirements, and expected unsupported-by-design boundaries.
• Do not expand the default baseline until the warning policy and durable evidence path are in place.
• Produce a short recommendation for the next maturity slice, likely navigation, query, ACL/policy, versioning/PWC, or change-log depth.

Progress:

• Added the next-coverage recommendation to docs/LOG-REVIEW.md.
• Recommended order is navigation/read-path depth first, metadata query second, ACL/policy discovery third, and versioning/PWC/change-log only after the product deliberately advertises those capabilities.
• Left the default baseline unchanged at repository-type plus object-content.

D3.8 - State Hub And Operator Docs

id: OPEN-CMIS-TCK-WP-0003-T008
status: done
priority: medium
state_hub_task_id: "32724628-d427-47c2-ac40-3f3f90e3a2b9"

Acceptance:

• Sync this workplan with the state hub.
• Update README/runbook references so operators know how to review warnings after a run.
• Make it clear that guide-board produces preparation evidence and operational readiness signals, not certification or audit assurance.
• Ensure doc updates cite the latest raw and persisted evidence baselines.

Progress:

• Added docs/LOG-REVIEW.md.
• Updated README, local runbook, and service/retention docs.
• Synced the completed workplan and task statuses into the state hub.

Definition Of Done

• Future local CMIS assessments keep raw evidence in a durable run location.
• HTTP transport warnings are policy-classified rather than manually explained after every run.
• The local in-memory pilot has either zero unexpected warnings or a documented expected-warning profile.
• A log-review report can be generated from any guide-board run directory.
• Skipped OpenCMIS cases are interpreted against advertised CMIS capability boundaries.
• The next coverage frontier is explicit and does not blur preparation evidence with formal certification.