diff --git a/README.md b/README.md index 62973a8..b800389 100644 --- a/README.md +++ b/README.md @@ -73,6 +73,8 @@ PYTHONPATH=../guide-board/src python3 -m unittest discover -s tests ## Docs - [docs/CMIS-PROFILES.md](docs/CMIS-PROFILES.md) +- [docs/CONTAINER-HANDOFF.md](docs/CONTAINER-HANDOFF.md) +- [docs/LOCAL-RUNBOOK.md](docs/LOCAL-RUNBOOK.md) - [docs/LOCAL-TCK-RUNTIME.md](docs/LOCAL-TCK-RUNTIME.md) - [docs/OPENCMIS-TCK-RUNNER.md](docs/OPENCMIS-TCK-RUNNER.md) - [docs/SERVICE-AND-RETENTION.md](docs/SERVICE-AND-RETENTION.md) diff --git a/docs/CONTAINER-HANDOFF.md b/docs/CONTAINER-HANDOFF.md new file mode 100644 index 0000000..22ca49f --- /dev/null +++ b/docs/CONTAINER-HANDOFF.md @@ -0,0 +1,74 @@ +# Container And Test Host Handoff + +Status: draft +Created: 2026-05-08 + +## Purpose + +This note captures what should move into a container or dedicated test host +after local OpenCMIS TCK execution is stable. + +Containerization is not a prerequisite for WP-0002 local execution. + +## Candidate Runtime Image + +The first dedicated runtime image should include: + +- Python 3.11+ +- Java runtime/JDK verified against OpenCMIS TCK 1.1.0 +- Maven +- guide-board core +- `open-cmis-tck` extension + +It should not bake in secrets or target-specific profiles. + +## Mounts + +Expected mounts: + +```text +/profiles target and assessment profiles +/credentials optional credential files +/runs generated guide-board run directories +/m2 optional Maven cache +``` + +For licensed or externally supplied assets, add: + +```text +/assets +``` + +The current OpenCMIS TCK runtime uses Maven Central and does not require +vendored assets. + +## Environment + +Useful environment variables: + +```text +CMIS_TCK_USER +CMIS_TCK_PASSWORD +MAVEN_OPTS +GUIDE_BOARD_EXTENSION_PATHS +``` + +The adapter should still prefer profile-level `credentials_ref` over implicit +environment assumptions. + +## Network Assumptions + +The test host must reach: + +- the CMIS Browser Binding endpoint under test, +- Maven Central or an internal Maven mirror during dependency resolution, +- any internal credential source if file mounts are not used. + +## Handoff Criteria + +Move to a container or test host after: + +- `scripts/bootstrap_opencmis_tck.py --resolve` is ready locally, +- at least one repository/type live run has produced guide-board evidence, +- real TCK output normalization has been reviewed, +- the maturity scorecard has its first evidence-backed output. diff --git a/docs/LOCAL-RUNBOOK.md b/docs/LOCAL-RUNBOOK.md new file mode 100644 index 0000000..035b626 --- /dev/null +++ b/docs/LOCAL-RUNBOOK.md @@ -0,0 +1,167 @@ +# Local OpenCMIS TCK Runbook + +Status: draft +Created: 2026-05-08 + +## Purpose + +This runbook is the workstation path from a clean checkout to the first local +OpenCMIS TCK run through guide-board. + +The local path has two stages: + +1. Prove the guide-board/OpenCMIS adapter wiring with the dry-run profile. +2. Install Java/Maven, resolve the OpenCMIS TCK runtime, and run the real + ConsoleRunner profile. + +## Prerequisites + +The extension and guide-board repositories should be siblings: + +```text +/home/worsch/guide-board +/home/worsch/open-cmis-tck +``` + +The guide-board core is used from source: + +```sh +cd /home/worsch/guide-board +PYTHONPATH=src python3 -m guide_board --extension-dir ../open-cmis-tck extensions validate +``` + +## Dry-Run Adapter Check + +The dry-run profile verifies profile resolution, CMIS preflight, wrapper +invocation, ConsoleRunner adapter argument expansion, session file generation, +artifact references, mapping, and report creation. It does not require Java or +Maven. + +Start or point to a CMIS Browser Binding endpoint, then run: + +```sh +cd /home/worsch/guide-board +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + run \ + --target ../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json \ + --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-local-dry-run.json \ + --output-dir /tmp/open-cmis-tck-dry-run +``` + +Expected dry-run artifacts: + +```text +/tmp/open-cmis-tck-dry-run/reports/report.md +/tmp/open-cmis-tck-dry-run/reports/assessment-package.json +/tmp/open-cmis-tck-dry-run/normalized/evidence.json +/tmp/open-cmis-tck-dry-run/artifacts/open-cmis-tck/tck/repository-type/session.properties.redacted +/tmp/open-cmis-tck-dry-run/artifacts/open-cmis-tck/tck/repository-type/groups.txt +``` + +If preflight fails, fix the target profile or endpoint before continuing. + +## Install Java And Maven + +The current WSL environment needs Java and Maven before the real TCK can run: + +```sh +sudo apt-get update +sudo apt-get install -y openjdk-17-jdk maven +``` + +Use a managed local Java/Maven installation instead if preferred. The bootstrap +only requires `java` and `mvn` on `PATH`. + +## Resolve The TCK Runtime + +```sh +cd /home/worsch/open-cmis-tck +PYTHONPATH=src python3 scripts/bootstrap_opencmis_tck.py --resolve +``` + +This resolves: + +```text +org.apache.chemistry.opencmis:chemistry-opencmis-test-tck:1.1.0 +``` + +into the local Maven cache and writes: + +```text +.local/opencmis-tck/runtime-summary.json +``` + +The `.local/` directory is ignored and must not be committed. + +## Real TCK Run + +After bootstrap reports `ready`, run the baseline assessment: + +```sh +cd /home/worsch/guide-board +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + run \ + --target ../open-cmis-tck/profiles/targets/kontextual-cmis-compat.json \ + --assessment ../open-cmis-tck/profiles/assessments/cmis-browser-baseline.json \ + --output-dir /tmp/open-cmis-tck-live +``` + +The baseline currently selects: + +- `repository-type` +- `object-content` + +Expand selected check groups only after the repository/type run produces useful +output. + +## Authenticated Targets + +For environment credentials: + +```sh +export CMIS_TCK_USER='cmis-user' +export CMIS_TCK_PASSWORD='local-secret' +``` + +Use a target profile with: + +```json +"credentials_ref": "env:CMIS_TCK_USER,CMIS_TCK_PASSWORD" +``` + +The adapter uses a private session file during execution and retains only a +redacted session file as an artifact. + +## Troubleshooting + +`java is not available on PATH`: + +Install a JDK or expose an existing JDK to WSL. + +`maven is not available on PATH`: + +Install Maven or expose an existing Maven executable to WSL. + +Maven dependency resolution fails: + +Check network access to Maven Central or rerun with a populated Maven cache. + +Preflight returns `infrastructure_error`: + +Check endpoint URL, server process, firewall, credentials, and timeout. + +Repository ID mismatch: + +Use the repository IDs reported by preflight and update +`runtime_policy.opencmis_tck.repository_id`. + +TCK command times out: + +Increase `runtime_policy.timeout_seconds` or narrow selected check groups. + +Formal boundary: + +These runs generate preparation evidence only. They do not issue CMIS +certification. diff --git a/docs/LOCAL-TCK-RUNTIME.md b/docs/LOCAL-TCK-RUNTIME.md index a57e9a9..df56eaf 100644 --- a/docs/LOCAL-TCK-RUNTIME.md +++ b/docs/LOCAL-TCK-RUNTIME.md @@ -76,6 +76,8 @@ one outside WSL. ## Guide-Board Invocation +For the full local sequence, see `docs/LOCAL-RUNBOOK.md`. + The baseline assessment profile now points the OpenCMIS wrapper at: ```text diff --git a/profiles/assessments/cmis-browser-local-dry-run.json b/profiles/assessments/cmis-browser-local-dry-run.json new file mode 100644 index 0000000..3d80895 --- /dev/null +++ b/profiles/assessments/cmis-browser-local-dry-run.json @@ -0,0 +1,59 @@ +{ + "id": "cmis-browser-local-dry-run", + "framework_refs": [ + "cmis.browser-binding.compatibility.v1" + ], + "extension_refs": [ + "open-cmis-tck" + ], + "target_profile_ref": "kontextual-cmis-compat", + "selected_check_groups": { + "open-cmis-tck": [ + "repository-type" + ] + }, + "expectations_ref": null, + "waivers_ref": null, + "output_policy": { + "report_formats": [ + "json", + "markdown" + ], + "artifact_retention": "raw-logs-plus-summary" + }, + "retention_policy": { + "summary_days": 365, + "raw_artifact_days": 30 + }, + "runtime_policy": { + "offline": false, + "timeout_seconds": 60, + "opencmis_tck": { + "repository_id": "compat-tck", + "requires_java_maven": false, + "command": [ + "python3", + "{extension_path}/adapters/opencmis_console_adapter.py", + "--browser-url", + "{browser_url}", + "--repository-id", + "{repository_id}", + "--check-group", + "{check_group}", + "--artifact-dir", + "{artifact_dir}", + "--run-dir", + "{run_dir}", + "--extension-path", + "{extension_path}", + "--credentials-ref", + "{credentials_ref}", + "--target-profile-dir", + "{target_profile_dir}", + "--timeout-seconds", + "{timeout_seconds}", + "--dry-run" + ] + } + } +} diff --git a/runners/opencmis_tck.py b/runners/opencmis_tck.py index 52ad155..ab93e77 100644 --- a/runners/opencmis_tck.py +++ b/runners/opencmis_tck.py @@ -246,6 +246,9 @@ def _normalize_json_result( selected_group: str | None, ) -> dict[str, Any]: artifact_refs = _artifact_refs_from_payload(payload) + source_facts = payload.get("facts", {}) + if not isinstance(source_facts, dict): + source_facts = {} cases = _json_cases(payload) if cases: counts: dict[str, int] = {} @@ -266,6 +269,7 @@ def _normalize_json_result( f"OpenCMIS TCK group {selected_group!r} produced {sum(counts.values())} normalized case result(s)." ], "facts": { + **source_facts, "normalizer": "json-cases", "result_counts": counts, "cases": normalized_cases[:200], @@ -280,6 +284,7 @@ def _normalize_json_result( "result": result, "observations": _observations_from_payload(payload, selected_group), "facts": { + **source_facts, "normalizer": "json-runner-result", "result_counts": {result: 1}, "payload": payload, diff --git a/tests/test_open_cmis_tck.py b/tests/test_open_cmis_tck.py index 3f4327d..64ac3ad 100644 --- a/tests/test_open_cmis_tck.py +++ b/tests/test_open_cmis_tck.py @@ -474,6 +474,101 @@ class OpenCmisTckExtensionTests(unittest.TestCase): thread.join(timeout=5) server.server_close() + def test_guide_board_dry_run_invokes_console_adapter_and_captures_artifacts(self) -> None: + server = HTTPServer(("127.0.0.1", 0), _CmisHandler) + thread = threading.Thread(target=server.serve_forever) + thread.daemon = True + thread.start() + try: + with TemporaryDirectory() as temporary_directory: + temp_root = Path(temporary_directory) + target_path = temp_root / "target.json" + assessment_path = temp_root / "assessment.json" + _write_target(target_path, server.server_port, "local-cmis-guide-dry-run") + _write_assessment( + assessment_path, + "local-cmis-guide-dry-run", + "local-cmis-guide-dry-run", + ["repository-type"], + None, + { + "requires_java_maven": False, + "repository_id": "local-test-repository", + "command": [ + sys.executable, + str(ROOT / "adapters" / "opencmis_console_adapter.py"), + "--browser-url", + "{browser_url}", + "--repository-id", + "{repository_id}", + "--check-group", + "{check_group}", + "--artifact-dir", + "{artifact_dir}", + "--run-dir", + "{run_dir}", + "--extension-path", + "{extension_path}", + "--credentials-ref", + "{credentials_ref}", + "--target-profile-dir", + "{target_profile_dir}", + "--timeout-seconds", + "{timeout_seconds}", + "--dry-run", + ], + }, + ) + + result = run_assessment( + CORE_ROOT, + target_path, + assessment_path, + temp_root / "run", + [ROOT], + ) + run_dir = Path(result["run_dir"]) + evidence = json.loads( + (run_dir / "normalized" / "evidence.json").read_text(encoding="utf-8") + )["evidence"] + package = json.loads( + (run_dir / "reports" / "assessment-package.json").read_text( + encoding="utf-8" + ) + ) + artifact_paths = { + item["path"] for item in package["artifact_manifest"] + } + + self.assertEqual(result["status"], "completed") + self.assertEqual(evidence[1]["result"], "skipped") + self.assertEqual( + evidence[1]["facts"]["adapter"], + "opencmis-console-runner", + ) + self.assertIn( + "artifacts/open-cmis-tck/tck/repository-type/session.properties.redacted", + artifact_paths, + ) + self.assertIn( + "artifacts/open-cmis-tck/tck/repository-type/groups.txt", + artifact_paths, + ) + self.assertFalse( + ( + run_dir + / "artifacts" + / "open-cmis-tck" + / "tck" + / "repository-type" + / "session-private.properties" + ).exists() + ) + finally: + server.shutdown() + thread.join(timeout=5) + server.server_close() + def test_guide_board_service_runs_cmis_extension(self) -> None: server = HTTPServer(("127.0.0.1", 0), _CmisHandler) thread = threading.Thread(target=server.serve_forever) diff --git a/workplans/OPEN-CMIS-TCK-WP-0002-live-test-infrastructure.md b/workplans/OPEN-CMIS-TCK-WP-0002-live-test-infrastructure.md index 8524156..be35863 100644 --- a/workplans/OPEN-CMIS-TCK-WP-0002-live-test-infrastructure.md +++ b/workplans/OPEN-CMIS-TCK-WP-0002-live-test-infrastructure.md @@ -125,6 +125,9 @@ Progress: - The bootstrap writes `.local/opencmis-tck/runtime-summary.json` and can optionally resolve Maven dependencies with `--resolve`. - Current WSL posture is blocked because `java` and `mvn` are not on `PATH`. +- Added `docs/LOCAL-RUNBOOK.md` with the local sequence from dry-run adapter + check through Java/Maven install, Maven dependency resolution, and first real + guide-board run. ## D2.3 - OpenCMIS TCK Adapter Invocation @@ -153,6 +156,11 @@ Progress: directory. - The baseline assessment profile now points the OpenCMIS wrapper at this local adapter command. +- Added `profiles/assessments/cmis-browser-local-dry-run.json`, which exercises + guide-board, preflight, wrapper, ConsoleRunner adapter, and assessment-package + artifact capture without requiring Java/Maven. +- The guide-board dry-run captures redacted session properties and group lists + as fingerprinted assessment artifacts. - Live execution remains blocked until Java/Maven are installed and Maven can resolve the TCK runtime. @@ -247,7 +255,7 @@ Acceptance: ```task id: OPEN-CMIS-TCK-WP-0002-T008 -status: todo +status: done priority: medium state_hub_task_id: "da7022de-3d05-43bf-9c06-91f00c159bdc" ``` @@ -260,11 +268,18 @@ Acceptance: - Provide copy-pasteable commands for local execution through guide-board. - Clearly mark the formal-certification boundary. +Progress: + +- Added `docs/LOCAL-RUNBOOK.md` with dry-run, Java/Maven install, TCK runtime + resolution, real guide-board run, authenticated target, and troubleshooting + commands. +- Linked the runbook from README and local runtime documentation. + ## D2.9 - Container/Test Host Handoff Notes ```task id: OPEN-CMIS-TCK-WP-0002-T009 -status: todo +status: done priority: low state_hub_task_id: "254d099a-5406-43b0-9497-9af594a9b911" ``` @@ -278,6 +293,12 @@ Acceptance: - Do not implement the container/test-host move in this workplan unless local execution is already proven. +Progress: + +- Added `docs/CONTAINER-HANDOFF.md` with runtime-image contents, mounts, + environment variables, network assumptions, and local-readiness handoff + criteria. + ## Definition Of Done - A developer can run one documented local bootstrap command.