generated from coulomb/repo-seed
327 lines
12 KiB
Markdown
327 lines
12 KiB
Markdown
---
|
|
id: OPEN-CMIS-TCK-WP-0003
|
|
type: extension-workplan
|
|
title: "Assessment Log Review And Hardening"
|
|
repo: open-cmis-tck
|
|
extension: open-cmis-tck
|
|
domain: markitect
|
|
status: completed
|
|
owner: codex
|
|
planning_priority: high
|
|
planning_order: 4
|
|
created: "2026-05-14"
|
|
updated: "2026-05-14"
|
|
depends_on:
|
|
- "OPEN-CMIS-TCK-WP-0002"
|
|
state_hub_workstream_id: "5711ee2f-eaa9-428a-a4b2-e7383bfbf18a"
|
|
---
|
|
|
|
# OPEN-CMIS-TCK-WP-0003: Assessment Log Review And Hardening
|
|
|
|
## Purpose
|
|
|
|
Use the first real `kontextual-engine` OpenCMIS TCK assessment runs to harden
|
|
the `open-cmis-tck` guide-board extension around warning policy, durable
|
|
evidence retention, and repeatable log review.
|
|
|
|
The latest `kontextual-engine` release-readiness run is healthy for the selected
|
|
Browser Binding baseline: no hard TCK failures, no infrastructure errors, no
|
|
unexpected findings, and empty stderr artifacts. The remaining work is mostly
|
|
facility maturity: make the one current warning intentional, preserve raw
|
|
evidence outside ephemeral `/tmp` paths, and give future assessments a compact
|
|
"what should we fix next" report instead of relying on manual `rg` passes.
|
|
|
|
## Evidence Reviewed
|
|
|
|
- Latest release-readiness evidence:
|
|
`/home/worsch/kontextual-engine/docs/cmis-opencmis-tck-release-readiness-evidence-2026-05-13T223537Z.md`
|
|
- Latest raw release-readiness run:
|
|
`/tmp/kontextual-cmis-release-20260514-toolchain`
|
|
- Prior raw run before `appendContentStream()` support:
|
|
`/tmp/open-cmis-tck-kontextual-20260513T230205Z`
|
|
- Earlier implementation evidence:
|
|
`/home/worsch/kontextual-engine/docs/cmis-opencmis-tck-implementation-evidence-2026-05-08T092113Z.md`
|
|
- Local extension self-test run:
|
|
`/home/worsch/open-cmis-tck/.local/runs/opencmis-inmemory-pilot`
|
|
- Local OpenCMIS in-memory server logs:
|
|
`/home/worsch/open-cmis-tck/.local/opencmis-inmemory/logs`
|
|
|
|
## Current Findings
|
|
|
|
1. The latest `kontextual-engine` run completed with Guide Board summary
|
|
`pass: 2`, `warning: 1`, `unexpected_findings: 0`.
|
|
2. `console-runner-stderr.txt` and `stderr.log` are empty for both selected TCK
|
|
groups in the latest run.
|
|
3. The remaining current warning is from OpenCMIS
|
|
`SecurityTest.java:67`: `HTTPS is not used. Credentials might be transferred
|
|
as plain text!`
|
|
4. The previous `appendContentStream()` warning in
|
|
`SetAndDeleteContentTest.java:200` is closed in the latest run.
|
|
5. The latest object/content run still has skipped cases for non-creatable
|
|
relationship, policy, and item types, plus folder-name change-token subcases.
|
|
These align with declared capability boundaries and are not errors, but they
|
|
should remain visible as maturity scope.
|
|
6. The local OpenCMIS in-memory pilot has two repository/type warnings:
|
|
loopback HTTP and `Thin client URI is not set!`. Tomcat and in-memory server
|
|
logs did not show warning/error/exception lines in the scan.
|
|
7. Evidence retention is fragile: several useful raw runs live under `/tmp`, and
|
|
at least one earlier `/tmp` run was already unavailable when later evidence
|
|
was written.
|
|
|
|
## Boundary
|
|
|
|
This workplan hardens the `open-cmis-tck` extension and its assessment
|
|
operations. Product changes for `kontextual-engine` belong in that repository.
|
|
This workplan may document product-facing follow-up candidates, but it should
|
|
not modify the product repo directly.
|
|
|
|
## D3.1 - Capture Current Log Triage Baseline
|
|
|
|
```task
|
|
id: OPEN-CMIS-TCK-WP-0003-T001
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "1a262cad-a945-4a93-a957-02f2fdb497f1"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Inspect the latest persisted evidence and raw run artifacts for
|
|
`kontextual-engine`.
|
|
- Separate current findings from older findings that have already been closed.
|
|
- Inspect the local in-memory pilot logs so extension self-test warnings are not
|
|
confused with product warnings.
|
|
- Record the baseline in this workplan.
|
|
|
|
Progress:
|
|
|
|
- Confirmed the latest raw release-readiness run has no `fail`,
|
|
`infrastructure_error`, unexpected finding, stderr output, or exception trace.
|
|
- Confirmed the only current `kontextual-engine` TCK warning is local HTTP
|
|
transport.
|
|
- Confirmed `appendContentStream()` was a warning in the prior raw run and is
|
|
gone in the latest raw run.
|
|
- Confirmed the local in-memory pilot still reports loopback HTTP and missing
|
|
thin-client URI warnings, while server logs are clean.
|
|
|
|
## D3.2 - Durable Assessment Archive Path
|
|
|
|
```task
|
|
id: OPEN-CMIS-TCK-WP-0003-T002
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "1e31b306-f21e-4bac-8e69-56d586d6712e"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Provide a recommended non-ephemeral output layout for local product
|
|
assessments, for example `.local/runs/<target>/<run-id>` or a configured
|
|
workspace archive path.
|
|
- Add an operator command or documented copy/import step that preserves raw
|
|
TCK stdout/stderr, normalized evidence, findings, mappings, run metadata,
|
|
report, scorecard, and artifact manifest before `/tmp` cleanup can remove
|
|
them.
|
|
- Preserve artifact hashes or package metadata so copied evidence remains
|
|
auditable.
|
|
- Update the local runbook and service/retention notes with the durable path.
|
|
|
|
Progress:
|
|
|
|
- Added `src/open_cmis_tck/archive.py` and
|
|
`scripts/archive_assessment_run.py`.
|
|
- The archive command copies a run into `.local/runs/archive/<target>/<run-id>`
|
|
by default and writes `archive-manifest.json` with SHA-256 hashes, file
|
|
sizes, source path, archive path, run ID, target profile reference, and
|
|
assessment profile reference.
|
|
- Updated README, local runbook, and service/retention docs with the archive
|
|
command.
|
|
- Archived the latest kontextual release-readiness run to
|
|
`.local/runs/archive/kontextual-cmis-compat/run-20260513T223537Z`.
|
|
|
|
## D3.3 - Warning Policy And HTTPS Deployment Gate
|
|
|
|
```task
|
|
id: OPEN-CMIS-TCK-WP-0003-T003
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "58d2cf64-0db6-4f9a-a8d5-df9ef870557b"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Define how warning policy distinguishes local loopback test topology from a
|
|
deployment or release gate.
|
|
- Treat the OpenCMIS HTTP warning as acceptable only for explicit local
|
|
loopback profiles or documented local waivers.
|
|
- Make non-loopback or release-target HTTP warnings visible as deployment
|
|
blockers, even when the TCK group return code is `0`.
|
|
- Record warning policy in a profile, expectation, or waiver file rather than
|
|
burying it in narrative evidence.
|
|
|
|
Progress:
|
|
|
|
- Added `profiles/expectations/opencmis-warning-policy.json`.
|
|
- Classified the OpenCMIS HTTP warning as accepted only for local/test loopback
|
|
HTTP endpoints.
|
|
- Non-loopback or production-like HTTP warnings are now classified as
|
|
`deployment_transport_blocker` by the log-review command.
|
|
|
|
## D3.4 - OpenCMIS In-Memory Pilot Warning Cleanup
|
|
|
|
```task
|
|
id: OPEN-CMIS-TCK-WP-0003-T004
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "12331abc-c28f-4140-9a04-a70eb761bccf"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Investigate whether the local OpenCMIS in-memory server can expose a
|
|
`thinClientURI` through configuration.
|
|
- If the upstream in-memory server cannot be configured cleanly, mark the
|
|
warning as an expected self-test limitation with a precise source location and
|
|
explanation.
|
|
- Keep the in-memory pilot useful as an extension smoke test without making its
|
|
target-specific warnings look like guide-board defects.
|
|
- Document the expected warning posture in the local runbook.
|
|
|
|
Progress:
|
|
|
|
- Added an explicit policy entry for the `opencmis-inmemory-local`
|
|
`Thin client URI is not set!` warning.
|
|
- The in-memory pilot review now classifies loopback HTTP and missing
|
|
thin-client URI as accepted local self-test warnings.
|
|
- Optional external server-log findings are reported as context without
|
|
changing the run status by themselves, because those log directories may
|
|
include historical startup attempts outside the assessed run.
|
|
|
|
## D3.5 - Automated Log Review Report
|
|
|
|
```task
|
|
id: OPEN-CMIS-TCK-WP-0003-T005
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "e445f909-678b-4236-a025-d5913e5473ed"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Add a command that scans a guide-board run directory for OpenCMIS stdout,
|
|
stderr, normalized results, findings, and known server logs.
|
|
- Generate `reports/opencmis-log-review.json` and
|
|
`reports/opencmis-log-review.md`.
|
|
- Highlight hard errors, non-empty stderr, new warnings, known accepted
|
|
warnings, skipped cases, unexpected findings, and closed-warning comparisons
|
|
when a previous run is supplied.
|
|
- Add regression tests with sanitized fixtures for the current HTTP warning,
|
|
the now-closed `appendContentStream()` warning, empty stderr, and skipped
|
|
capability-boundary cases.
|
|
|
|
Progress:
|
|
|
|
- Added `src/open_cmis_tck/log_review.py` and
|
|
`scripts/opencmis_log_review.py`.
|
|
- The command writes `reports/opencmis-log-review.json` and
|
|
`reports/opencmis-log-review.md`.
|
|
- Verified it against `/tmp/kontextual-cmis-release-20260514-toolchain` with
|
|
`/tmp/open-cmis-tck-kontextual-20260513T230205Z` as the previous run.
|
|
- Added regression coverage for accepted HTTP warnings, non-loopback deployment
|
|
blockers, closed append warnings, stderr handling, and skipped capability
|
|
boundaries.
|
|
|
|
## D3.6 - Skip And Capability Boundary Interpretation
|
|
|
|
```task
|
|
id: OPEN-CMIS-TCK-WP-0003-T006
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "6df3e8c5-c2d2-4eb3-973e-76644ef7ee8f"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Group skipped OpenCMIS cases by declared repository capability or type
|
|
creatability boundary.
|
|
- Distinguish "expected because the capability is not advertised" from "skipped
|
|
because the target could not exercise an advertised capability."
|
|
- Keep skipped cases visible in reports and maturity scorecards without treating
|
|
them as failures when they match the advertised capability profile.
|
|
- Add coverage for relationship, policy, item, type-subtype, and folder-name
|
|
change-token skips seen in the latest raw run.
|
|
|
|
Progress:
|
|
|
|
- Added skip-boundary classification in the log-review report.
|
|
- Current expected skip rules cover relationship, policy, item, document
|
|
subtype, and folder-name mutation cases.
|
|
- If the target advertises the required capability and OpenCMIS still skips the
|
|
case, the review becomes `review_required`.
|
|
|
|
## D3.7 - Next Coverage Frontier
|
|
|
|
```task
|
|
id: OPEN-CMIS-TCK-WP-0003-T007
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "f793e1b8-a2b1-4f9b-9972-b6b18d1ba56a"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Identify which additional OpenCMIS TCK groups are realistic after the current
|
|
repository/type and object/content baseline.
|
|
- For each candidate group, record target preconditions, likely product
|
|
capability requirements, and expected unsupported-by-design boundaries.
|
|
- Do not expand the default baseline until the warning policy and durable
|
|
evidence path are in place.
|
|
- Produce a short recommendation for the next maturity slice, likely navigation,
|
|
query, ACL/policy, versioning/PWC, or change-log depth.
|
|
|
|
Progress:
|
|
|
|
- Added the next-coverage recommendation to `docs/LOG-REVIEW.md`.
|
|
- Recommended order is navigation/read-path depth first, metadata query second,
|
|
ACL/policy discovery third, and versioning/PWC/change-log only after the
|
|
product deliberately advertises those capabilities.
|
|
- Left the default baseline unchanged at `repository-type` plus
|
|
`object-content`.
|
|
|
|
## D3.8 - State Hub And Operator Docs
|
|
|
|
```task
|
|
id: OPEN-CMIS-TCK-WP-0003-T008
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "32724628-d427-47c2-ac40-3f3f90e3a2b9"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Sync this workplan with the state hub.
|
|
- Update README/runbook references so operators know how to review warnings
|
|
after a run.
|
|
- Make it clear that guide-board produces preparation evidence and operational
|
|
readiness signals, not certification or audit assurance.
|
|
- Ensure doc updates cite the latest raw and persisted evidence baselines.
|
|
|
|
Progress:
|
|
|
|
- Added `docs/LOG-REVIEW.md`.
|
|
- Updated README, local runbook, and service/retention docs.
|
|
- Synced the completed workplan and task statuses into the state hub.
|
|
|
|
## Definition Of Done
|
|
|
|
- Future local CMIS assessments keep raw evidence in a durable run location.
|
|
- HTTP transport warnings are policy-classified rather than manually explained
|
|
after every run.
|
|
- The local in-memory pilot has either zero unexpected warnings or a documented
|
|
expected-warning profile.
|
|
- A log-review report can be generated from any guide-board run directory.
|
|
- Skipped OpenCMIS cases are interpreted against advertised CMIS capability
|
|
boundaries.
|
|
- The next coverage frontier is explicit and does not blur preparation evidence
|
|
with formal certification.
|