From cc43881565c3bfc50d13e545a23e9eed761b2654 Mon Sep 17 00:00:00 2001 From: tegwick Date: Thu, 7 May 2026 22:58:28 +0200 Subject: [PATCH] chore(consistency): sync task status from DB [auto] Updated by fix-consistency on 2026-05-07: - update .custodian-brief.md for open-cmis-tck --- .custodian-brief.md | 15 +- README.md | 16 +- docs/CMIS-PROFILES.md | 67 ++++ docs/OPENCMIS-TCK-RUNNER.md | 97 +++++ docs/SERVICE-AND-RETENTION.md | 56 +++ extension.json | 29 +- mappings/cmis-capability-map.json | 28 ++ .../assessments/cmis-browser-baseline.json | 6 +- runners/opencmis_tck.py | 371 +++++++++++++++++- src/open_cmis_tck/preflight.py | 155 +++++++- src/open_cmis_tck/profile.py | 146 +++++++ tests/test_open_cmis_tck.py | 290 +++++++++++++- ...PEN-CMIS-TCK-WP-0001-harness-foundation.md | 64 ++- 13 files changed, 1301 insertions(+), 39 deletions(-) create mode 100644 docs/CMIS-PROFILES.md create mode 100644 docs/OPENCMIS-TCK-RUNNER.md create mode 100644 docs/SERVICE-AND-RETENTION.md create mode 100644 src/open_cmis_tck/profile.py diff --git a/.custodian-brief.md b/.custodian-brief.md index af3eb05..5a508a3 100644 --- a/.custodian-brief.md +++ b/.custodian-brief.md @@ -2,12 +2,23 @@ # Custodian Brief — open-cmis-tck **Domain:** markitect -**Last synced:** 2026-05-07 20:34 UTC +**Last synced:** 2026-05-07 20:58 UTC **State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)* ## Active Workstreams -*(none — repo may need first-session setup)* +### Live OpenCMIS TCK Execution And Capability Maturity +Progress: 0/9 done | workstream_id: `da3f0d16-ba8e-4147-b0fc-ab3462e0b7b0` + +**Open tasks:** +- · D2.1 - Resolve TCK Runtime And Access Model `f3144edb` +- · D2.2 - Local Environment Bootstrap Command `f993c1ef` +- · D2.3 - OpenCMIS TCK Adapter Invocation `a446a80f` +- · D2.4 - Target Profiles And Credential References `c33a4d9a` +- · D2.5 - Real Result Normalization `03ba9506` +- · D2.6 - Live Pilot Run `d9eb9384` +- · D2.7 - CMIS Capability Maturity Scorecard `7365052f` +- … and 2 more open tasks --- ## MCP Orientation (when available) diff --git a/README.md b/README.md index 59f8898..b484a69 100644 --- a/README.md +++ b/README.md @@ -53,8 +53,14 @@ Expected current behavior: - CMIS Browser Binding preflight runs first. - Failed preflight blocks downstream TCK groups. -- The Java/Maven OpenCMIS wrapper reports structured blockers until the final - Apache Chemistry TCK invocation is configured. +- The Java/Maven OpenCMIS wrapper reports structured blockers until a local + Apache Chemistry TCK command is configured. +- If a command is configured, raw stdout/stderr and normalized runner output are + captured under the guide-board run directory. + +Runner command configuration lives in +`runtime_policy.opencmis_tck.command`. See +[docs/OPENCMIS-TCK-RUNNER.md](docs/OPENCMIS-TCK-RUNNER.md). ## Tests @@ -64,6 +70,12 @@ Run extension tests with the guide-board core on `PYTHONPATH`: PYTHONPATH=../guide-board/src python3 -m unittest discover -s tests ``` +## Docs + +- [docs/CMIS-PROFILES.md](docs/CMIS-PROFILES.md) +- [docs/OPENCMIS-TCK-RUNNER.md](docs/OPENCMIS-TCK-RUNNER.md) +- [docs/SERVICE-AND-RETENTION.md](docs/SERVICE-AND-RETENTION.md) + ## Boundary Apache Chemistry/OpenCMIS TCK dependencies may be restricted by upstream diff --git a/docs/CMIS-PROFILES.md b/docs/CMIS-PROFILES.md new file mode 100644 index 0000000..c8c8bf1 --- /dev/null +++ b/docs/CMIS-PROFILES.md @@ -0,0 +1,67 @@ +# CMIS Profiles + +Status: draft +Created: 2026-05-07 + +## Purpose + +`open-cmis-tck` uses guide-board target and assessment profiles without adding a +separate persisted profile format. This keeps the extension compatible with the +guide-board planner while still giving CMIS-specific diagnostics through +`open_cmis_tck.profile.validate_cmis_profile_config`. + +## Target Profile Fields + +The CMIS target profile uses the guide-board `target-profile` schema: + +- `subject_type`: use `cmis-browser-binding-endpoint`. +- `endpoints`: include one endpoint with `binding` set to `cmis-browser` and + `url` set to the Browser Binding service document URL. +- `credentials_ref`: use `null` for anonymous/local development targets, or a + secret reference for authenticated repositories. +- `declared_capabilities`: list the CMIS requirement refs the target claims to + support, such as `cmis.repository-info`, `cmis.type-definitions`, + `cmis.object-services`, `cmis.content-streams`, `cmis.query`, `cmis.acl`, and + `cmis.versioning`. +- `known_gaps`: list unsupported optional requirements with a stable gap ID, + requirement refs, reason, and status such as `unsupported_by_design`. + +## Assessment Runtime Fields + +Repository selection and harness execution settings live in the assessment +profile because they are run policy, not target identity: + +```json +{ + "runtime_policy": { + "offline": false, + "timeout_seconds": 300, + "opencmis_tck": { + "repository_id": "compat-tck", + "requires_java_maven": true, + "command": ["java", "-jar", "/assets/opencmis-tck-runner.jar"] + } + } +} +``` + +`repository_id` is optional for preflight. If omitted, preflight selects the +first repository from the Browser Binding service document. A real TCK command +usually needs it. + +`command` is optional. When absent, the wrapper reports +`tck_invocation_not_configured` as a structured, expected bootstrap blocker. + +## Diagnostics + +Use the extension helper from tests or local scripts: + +```python +from open_cmis_tck.profile import validate_cmis_profile_config + +diagnostics = validate_cmis_profile_config(target_profile, assessment_profile) +``` + +The result contains `status`, `diagnostics`, and the interpreted `cmis_config`. +Diagnostics are intentionally actionable: they point to the field that should be +changed and explain what the extension expects. diff --git a/docs/OPENCMIS-TCK-RUNNER.md b/docs/OPENCMIS-TCK-RUNNER.md new file mode 100644 index 0000000..c40d1ee --- /dev/null +++ b/docs/OPENCMIS-TCK-RUNNER.md @@ -0,0 +1,97 @@ +# OpenCMIS TCK Runner + +Status: draft +Created: 2026-05-07 + +## Purpose + +The runner wrapper at `runners/opencmis_tck.py` is the boundary between +guide-board and Apache Chemistry OpenCMIS TCK execution. It keeps Java/Maven +setup, harness command lines, raw logs, and result normalization inside this +extension. + +## Dependency Checks + +By default, the wrapper checks: + +- `java -version` +- `mvn -version` + +If either dependency is unavailable, the runner returns `blocked` evidence with +`blocked_reason: missing_dependency`. + +Set `runtime_policy.opencmis_tck.requires_java_maven` to `false` only for tests +or custom harness commands that do not use the local Java/Maven toolchain. + +## Command Configuration + +Configure a TCK command as an argv list: + +```json +{ + "runtime_policy": { + "opencmis_tck": { + "repository_id": "compat-tck", + "command": [ + "java", + "-jar", + "/assets/opencmis-tck-runner.jar", + "--url", + "{browser_url}", + "--repository", + "{repository_id}", + "--group", + "{check_group}", + "--output", + "{artifact_dir}" + ] + } + } +} +``` + +Supported placeholders: + +- `{browser_url}` +- `{repository_id}` +- `{check_group}` +- `{target_id}` +- `{run_dir}` +- `{artifact_dir}` + +The wrapper also accepts `OPENCMIS_TCK_COMMAND_JSON` as a JSON string array, or +`OPENCMIS_TCK_COMMAND` as a shell-like string that is split into argv. The final +command still runs without shell expansion. + +## Raw Artifacts + +For each selected check group, artifacts are written under: + +```text +artifacts/open-cmis-tck/tck// +``` + +Current artifacts: + +- `invocation.json` +- `stdout.log` +- `stderr.log` +- `normalized-runner-result.json` + +The guide-board core fingerprints these files in the assessment package artifact +manifest when they are referenced by the runner result. + +## Normalization + +The wrapper normalizes, in order: + +1. JSON written to stdout with a `tests`, `cases`, or `results` array. +2. JUnit-style XML files written directly into `{artifact_dir}`. +3. Exit code only, when no structured output is found. + +Case statuses normalize to guide-board result vocabulary: `pass`, `fail`, +`skipped`, `expected_gap`, `unsupported_by_design`, `infrastructure_error`, and +related core statuses. + +This is enough to run a real local TCK adapter while preserving raw logs for +future Apache Chemistry-specific parsing refinements. diff --git a/docs/SERVICE-AND-RETENTION.md b/docs/SERVICE-AND-RETENTION.md new file mode 100644 index 0000000..e6d2746 --- /dev/null +++ b/docs/SERVICE-AND-RETENTION.md @@ -0,0 +1,56 @@ +# Service And Retention Integration + +Status: draft +Created: 2026-05-07 + +## Local Service + +`open-cmis-tck` does not run its own service. It plugs into the guide-board +local API as an external extension: + +```sh +cd ../guide-board +PYTHONPATH=src python3 -m guide_board \ + --extension-dir ../open-cmis-tck \ + serve --host 127.0.0.1 --port 8080 +``` + +The guide-board service can then: + +- list `open-cmis-tck` from `GET /extensions`, +- build CMIS run plans with `POST /assessments/plan`, +- start CMIS runs with `POST /runs`, +- inspect jobs with `GET /runs/{job_id}`, +- fetch reports with `GET /runs/{job_id}/reports`. + +CLI execution remains the primary and most transparent path. The service is a +transport and job-tracking layer over the same runner contracts. + +## Retention + +CMIS runs use the guide-board run directory contract. Each run writes: + +- `run.json` +- `retention-summary.json` +- `plan.json` +- `normalized/evidence.json` +- `normalized/findings.json` +- `normalized/mappings.json` +- `reports/assessment-package.json` +- `reports/report.md` + +The sample assessment profile keeps summaries for 365 days and raw artifacts for +30 days: + +```json +{ + "retention_policy": { + "summary_days": 365, + "raw_artifact_days": 30 + } +} +``` + +Compact `retention-summary.json` files are suitable for guide-board trend +summaries and downstream CMIS capability scorecards without retaining unbounded +raw TCK logs. diff --git a/extension.json b/extension.json index bf171c6..9ad6252 100644 --- a/extension.json +++ b/extension.json @@ -55,6 +55,33 @@ "cmis.versioning" ], "runner_ref": "opencmis-tck" + }, + { + "id": "relationships", + "name": "Relationship Checks", + "check_type": "executable_harness", + "requirement_refs": [ + "cmis.relationships" + ], + "runner_ref": "opencmis-tck" + }, + { + "id": "change-log", + "name": "Change Log Checks", + "check_type": "executable_harness", + "requirement_refs": [ + "cmis.change-log" + ], + "runner_ref": "opencmis-tck" + }, + { + "id": "extension-gaps", + "name": "Extension And Known Gap Checks", + "check_type": "executable_harness", + "requirement_refs": [ + "cmis.extensions" + ], + "runner_ref": "opencmis-tck" } ], "preflight_runner": "cmis-browser-preflight", @@ -78,7 +105,7 @@ "--context", "{context_json}" ], - "description": "Checks Java/Maven availability and prepares the future Apache Chemistry OpenCMIS TCK invocation." + "description": "Checks Java/Maven availability, invokes a configured Apache Chemistry OpenCMIS TCK command, captures raw artifacts, and returns normalized evidence." } ], "normalizers": [ diff --git a/mappings/cmis-capability-map.json b/mappings/cmis-capability-map.json index d106a61..6aa9a4a 100644 --- a/mappings/cmis-capability-map.json +++ b/mappings/cmis-capability-map.json @@ -54,12 +54,40 @@ "label": "ACL And Policy", "description": "Access-control list and policy behavior where supported by the target profile." }, + { + "requirement_ref": "cmis.policies", + "target_type": "capability_group", + "target_id": "acl-policy", + "label": "ACL And Policy", + "description": "Policy object behavior where supported by the target profile." + }, { "requirement_ref": "cmis.versioning", "target_type": "capability_group", "target_id": "versioning", "label": "Versioning", "description": "Checkout, checkin, version series, and version-specific object behavior." + }, + { + "requirement_ref": "cmis.relationships", + "target_type": "capability_group", + "target_id": "relationships", + "label": "Relationships", + "description": "Relationship object and relationship navigation behavior where supported by the repository." + }, + { + "requirement_ref": "cmis.change-log", + "target_type": "capability_group", + "target_id": "change-log", + "label": "Change Log", + "description": "Change log capability, token behavior, and change event retrieval." + }, + { + "requirement_ref": "cmis.extensions", + "target_type": "capability_group", + "target_id": "extension-gaps", + "label": "Extensions And Known Gaps", + "description": "Explicitly scoped CMIS extensions, unsupported optional services, and compatibility gaps." } ] } diff --git a/profiles/assessments/cmis-browser-baseline.json b/profiles/assessments/cmis-browser-baseline.json index 9cf8ef3..756c049 100644 --- a/profiles/assessments/cmis-browser-baseline.json +++ b/profiles/assessments/cmis-browser-baseline.json @@ -28,6 +28,10 @@ }, "runtime_policy": { "offline": false, - "timeout_seconds": 300 + "timeout_seconds": 300, + "opencmis_tck": { + "repository_id": "compat-tck", + "requires_java_maven": true + } } } diff --git a/runners/opencmis_tck.py b/runners/opencmis_tck.py index f9903bc..6a9507d 100644 --- a/runners/opencmis_tck.py +++ b/runners/opencmis_tck.py @@ -1,17 +1,20 @@ #!/usr/bin/env python3 """OpenCMIS TCK wrapper boundary. -This wrapper intentionally stops before invoking Apache Chemistry. Its current -job is to prove the command-runner contract, verify local Java/Maven posture, and -return structured evidence that the actual TCK execution remains pending. +The wrapper owns extension-local orchestration only: dependency checks, optional +user-supplied TCK command execution, raw artifact capture, and normalization into +the guide-board runner result contract. """ from __future__ import annotations import argparse import json +import os import shutil +import shlex import subprocess +import xml.etree.ElementTree as ET from pathlib import Path from typing import Any @@ -23,6 +26,7 @@ def main() -> int: context = _load_context(Path(args.context)) selected_group = context["step"].get("check_group") + config = _opencmis_policy(context) dependency_results = { "java": _probe_command(["java", "-version"]), "maven": _probe_command(["mvn", "-version"]), @@ -33,7 +37,7 @@ def main() -> int: if not result["available"] ] - if missing: + if config.get("requires_java_maven", True) and missing: _emit( { "result": "blocked", @@ -52,6 +56,11 @@ def main() -> int: ) return 0 + command_template = _configured_command(config) + if command_template is not None: + _emit(_run_configured_tck(context, selected_group, dependency_results, command_template)) + return 0 + _emit( { "result": "blocked", @@ -62,7 +71,7 @@ def main() -> int: "blocked_reason": "tck_invocation_not_configured", "selected_check_group": selected_group, "dependencies": dependency_results, - "next_step": "Resolve the Maven artifact, classpath, TCK group mapping, and raw artifact capture contract.", + "next_step": "Configure runtime_policy.opencmis_tck.command or OPENCMIS_TCK_COMMAND_JSON with an argv list for the local Apache Chemistry TCK runner.", }, "artifact_refs": [], } @@ -108,6 +117,358 @@ def _probe_command(command: list[str]) -> dict[str, Any]: } +def _run_configured_tck( + context: dict[str, Any], + selected_group: str | None, + dependency_results: dict[str, dict[str, Any]], + command_template: list[str], +) -> dict[str, Any]: + run_dir = Path(context["run_dir"]) + artifact_dir = run_dir / "artifacts" / "open-cmis-tck" / "tck" / _safe_id(selected_group or "unknown") + artifact_dir.mkdir(parents=True, exist_ok=True) + + command = [_expand_arg(arg, context, selected_group, artifact_dir) for arg in command_template] + invocation_ref = _write_json_artifact( + run_dir, + artifact_dir, + "invocation.json", + { + "selected_check_group": selected_group, + "command": command, + "target_profile_id": context["target_profile"]["id"], + "repository_id": _repository_id(context), + "browser_binding_url": _browser_url(context), + }, + ) + + try: + completed = subprocess.run( + command, + cwd=Path(context["extension_path"]), + capture_output=True, + text=True, + timeout=_timeout_seconds(context), + check=False, + ) + except FileNotFoundError as exc: + return { + "result": "blocked", + "observations": [ + f"OpenCMIS TCK command could not start because {exc.filename!r} was not found." + ], + "facts": { + "blocked_reason": "missing_command", + "selected_check_group": selected_group, + "dependencies": dependency_results, + "command": command, + }, + "artifact_refs": [invocation_ref], + } + except subprocess.TimeoutExpired: + return { + "result": "infrastructure_error", + "observations": [ + f"OpenCMIS TCK command timed out after {_timeout_seconds(context)} seconds." + ], + "facts": { + "selected_check_group": selected_group, + "dependencies": dependency_results, + "command": command, + }, + "artifact_refs": [invocation_ref], + } + + stdout_ref = _write_text_artifact(run_dir, artifact_dir, "stdout.log", completed.stdout) + stderr_ref = _write_text_artifact(run_dir, artifact_dir, "stderr.log", completed.stderr) + normalized = _normalize_tck_output(completed, artifact_dir, selected_group) + normalized["facts"].update( + { + "selected_check_group": selected_group, + "dependencies": dependency_results, + "command": command, + "returncode": completed.returncode, + "browser_binding_url": _browser_url(context), + "repository_id": _repository_id(context), + } + ) + normalized_ref = _write_json_artifact( + run_dir, + artifact_dir, + "normalized-runner-result.json", + normalized, + ) + normalized["artifact_refs"].extend([invocation_ref, stdout_ref, stderr_ref, normalized_ref]) + return normalized + + +def _normalize_tck_output( + completed: subprocess.CompletedProcess[str], + artifact_dir: Path, + selected_group: str | None, +) -> dict[str, Any]: + parsed_stdout = _parse_json(completed.stdout) + if isinstance(parsed_stdout, dict): + return _normalize_json_result(parsed_stdout, completed.returncode, selected_group) + + junit_files = sorted(artifact_dir.glob("*.xml")) + if junit_files: + return _normalize_junit_result(junit_files[0], completed.returncode, selected_group) + + if completed.returncode == 0: + return { + "result": "pass", + "observations": [ + "OpenCMIS TCK command completed successfully, but no structured result payload was found." + ], + "facts": { + "normalizer": "exit-code", + "result_counts": {"pass": 1}, + }, + "artifact_refs": [], + } + return { + "result": "fail", + "observations": [ + "OpenCMIS TCK command exited with a non-zero status and no structured result payload was found." + ], + "facts": { + "normalizer": "exit-code", + "result_counts": {"fail": 1}, + }, + "artifact_refs": [], + } + + +def _normalize_json_result( + payload: dict[str, Any], + returncode: int, + selected_group: str | None, +) -> dict[str, Any]: + cases = _json_cases(payload) + if cases: + counts: dict[str, int] = {} + normalized_cases = [] + for case in cases: + status = _normalize_case_status(str(case.get("status", "unknown"))) + counts[status] = counts.get(status, 0) + 1 + normalized_cases.append( + { + "id": str(case.get("id", case.get("name", "unnamed"))), + "status": status, + "message": str(case.get("message", case.get("reason", ""))), + } + ) + return { + "result": _aggregate_result(counts, returncode), + "observations": [ + f"OpenCMIS TCK group {selected_group!r} produced {sum(counts.values())} normalized case result(s)." + ], + "facts": { + "normalizer": "json-cases", + "result_counts": counts, + "cases": normalized_cases[:200], + }, + "artifact_refs": [], + } + + result = _normalize_case_status(str(payload.get("result", "unknown"))) + if returncode != 0 and result in {"pass", "warning", "skipped"}: + result = "infrastructure_error" + return { + "result": result, + "observations": _observations_from_payload(payload, selected_group), + "facts": { + "normalizer": "json-runner-result", + "result_counts": {result: 1}, + "payload": payload, + }, + "artifact_refs": [], + } + + +def _normalize_junit_result( + path: Path, + returncode: int, + selected_group: str | None, +) -> dict[str, Any]: + tree = ET.parse(path) + root = tree.getroot() + suites = [root] if root.tag == "testsuite" else list(root.findall("testsuite")) + total = sum(int(suite.get("tests", "0")) for suite in suites) + failures = sum(int(suite.get("failures", "0")) for suite in suites) + errors = sum(int(suite.get("errors", "0")) for suite in suites) + skipped = sum(int(suite.get("skipped", "0")) for suite in suites) + passed = max(0, total - failures - errors - skipped) + counts = { + "pass": passed, + "fail": failures + errors, + "skipped": skipped, + } + counts = {key: value for key, value in counts.items() if value} + return { + "result": _aggregate_result(counts, returncode), + "observations": [ + f"OpenCMIS TCK group {selected_group!r} produced JUnit-style XML results." + ], + "facts": { + "normalizer": "junit-xml", + "result_counts": counts, + "junit_xml": str(path), + }, + "artifact_refs": [], + } + + +def _json_cases(payload: dict[str, Any]) -> list[dict[str, Any]]: + for key in ("tests", "cases", "results"): + value = payload.get(key) + if isinstance(value, list) and all(isinstance(item, dict) for item in value): + return value + return [] + + +def _aggregate_result(counts: dict[str, int], returncode: int) -> str: + if counts.get("infrastructure_error"): + return "infrastructure_error" + if counts.get("fail"): + return "fail" + if counts.get("pass"): + return "pass" + if counts.get("expected_gap"): + return "expected_gap" + if counts.get("unsupported_by_design"): + return "unsupported_by_design" + if counts.get("skipped"): + return "skipped" + return "infrastructure_error" if returncode else "unknown" + + +def _normalize_case_status(value: str) -> str: + normalized = value.strip().lower().replace("-", "_").replace(" ", "_") + if normalized in {"ok", "success", "passed"}: + return "pass" + if normalized in {"failure", "failed", "error"}: + return "fail" + if normalized in {"skip", "skipped"}: + return "skipped" + if normalized in {"expected_skip", "expected_gap"}: + return "expected_gap" + if normalized in {"unsupported", "unsupported_by_design"}: + return "unsupported_by_design" + if normalized in {"infra", "infrastructure_error"}: + return "infrastructure_error" + if normalized in { + "pass", + "fail", + "warning", + "manual", + "not_applicable", + "waiver_applied", + "blocked", + "unknown", + }: + return normalized + return "unknown" + + +def _observations_from_payload(payload: dict[str, Any], selected_group: str | None) -> list[str]: + observations = payload.get("observations") + if isinstance(observations, list): + return [str(item) for item in observations] + message = payload.get("message") + if isinstance(message, str) and message: + return [message] + return [f"OpenCMIS TCK group {selected_group!r} returned a structured result."] + + +def _opencmis_policy(context: dict[str, Any]) -> dict[str, Any]: + policy = context["assessment_profile"].get("runtime_policy", {}).get("opencmis_tck", {}) + return policy if isinstance(policy, dict) else {} + + +def _configured_command(config: dict[str, Any]) -> list[str] | None: + command = config.get("command") + if isinstance(command, list) and all(isinstance(item, str) and item for item in command): + return command + + env_json = os.environ.get("OPENCMIS_TCK_COMMAND_JSON") + if env_json: + parsed = json.loads(env_json) + if isinstance(parsed, list) and all(isinstance(item, str) and item for item in parsed): + return parsed + raise ValueError("OPENCMIS_TCK_COMMAND_JSON must be a JSON string array") + + env_command = os.environ.get("OPENCMIS_TCK_COMMAND") + if env_command: + return shlex.split(env_command) + return None + + +def _expand_arg( + value: str, + context: dict[str, Any], + selected_group: str | None, + artifact_dir: Path, +) -> str: + return ( + value.replace("{run_dir}", context["run_dir"]) + .replace("{artifact_dir}", str(artifact_dir)) + .replace("{browser_url}", _browser_url(context) or "") + .replace("{repository_id}", _repository_id(context) or "") + .replace("{check_group}", selected_group or "") + .replace("{target_id}", context["target_profile"]["id"]) + ) + + +def _browser_url(context: dict[str, Any]) -> str | None: + for endpoint in context["target_profile"].get("endpoints", []): + if endpoint.get("binding") == "cmis-browser": + return endpoint.get("url") + return None + + +def _repository_id(context: dict[str, Any]) -> str | None: + value = _opencmis_policy(context).get("repository_id") + return value if isinstance(value, str) else None + + +def _timeout_seconds(context: dict[str, Any]) -> float: + runtime_policy = context["assessment_profile"].get("runtime_policy", {}) + opencmis_policy = _opencmis_policy(context) + configured = opencmis_policy.get("timeout_seconds", runtime_policy.get("timeout_seconds", 300)) + if not isinstance(configured, (int, float)): + return 300.0 + return max(1.0, float(configured)) + + +def _write_text_artifact(run_dir: Path, artifact_dir: Path, name: str, value: str) -> str: + path = artifact_dir / name + path.write_text(value, encoding="utf-8") + return str(path.relative_to(run_dir)) + + +def _write_json_artifact( + run_dir: Path, + artifact_dir: Path, + name: str, + value: dict[str, Any], +) -> str: + path = artifact_dir / name + path.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n", encoding="utf-8") + return str(path.relative_to(run_dir)) + + +def _parse_json(value: str) -> Any: + try: + return json.loads(value) + except json.JSONDecodeError: + return None + + +def _safe_id(value: str) -> str: + return "".join(char if char.isalnum() or char in {"-", "_"} else "_" for char in value) + + def _emit(value: dict[str, Any]) -> None: print(json.dumps(value, indent=2, sort_keys=True)) diff --git a/src/open_cmis_tck/preflight.py b/src/open_cmis_tck/preflight.py index 5029b0c..aa970f1 100644 --- a/src/open_cmis_tck/preflight.py +++ b/src/open_cmis_tck/preflight.py @@ -117,12 +117,43 @@ def run(context: dict[str, Any]) -> dict[str, Any]: } facts["json_detected"] = True - facts.update(_repository_facts(parsed)) + repository_facts = _repository_facts(parsed, context) + facts.update(repository_facts) + unsupported = [ + item + for item in repository_facts.get("capability_posture", []) + if item.get("status") == "unsupported" + ] + expected_gaps = [ + item + for item in repository_facts.get("capability_posture", []) + if item.get("status") == "expected_gap" + ] + if unsupported: + return { + "result": "fail", + "observations": [ + "CMIS Browser Binding endpoint is reachable, but declared capabilities are not supported by repository capability flags.", + "Unsupported declared requirements: " + + ", ".join(item["requirement_ref"] for item in unsupported) + + ".", + ], + "facts": facts, + "artifact_refs": artifact_refs, + } + + observations = [ + "CMIS Browser Binding endpoint is reachable and returned parseable JSON." + ] + if expected_gaps: + observations.append( + "Unsupported optional capabilities were accepted as known gaps: " + + ", ".join(item["requirement_ref"] for item in expected_gaps) + + "." + ) return { "result": "pass", - "observations": [ - "CMIS Browser Binding endpoint is reachable and returned parseable JSON." - ], + "observations": observations, "facts": facts, "artifact_refs": artifact_refs, } @@ -150,29 +181,40 @@ def _parse_json(body: bytes) -> Any: return None -def _repository_facts(value: Any) -> dict[str, Any]: +def _repository_facts(value: Any, context: dict[str, Any]) -> dict[str, Any]: if not isinstance(value, dict): return {"repository_shape": "unknown"} if "repositoryId" in value: + repository_id = str(value["repositoryId"]) return { "repository_shape": "single-repository-info", - "repository_ids": [value["repositoryId"]], + "repository_ids": [repository_id], + "selected_repository_id": repository_id, "cmis_version_supported": value.get("cmisVersionSupported"), "capabilities_present": isinstance(value.get("capabilities"), dict), + "capability_flags": _capability_flags(value), + "capability_posture": _capability_posture(value, context), } - repository_ids = [] + repositories: dict[str, dict[str, Any]] = {} for key, child in value.items(): if isinstance(child, dict) and ( "repositoryId" in child or "repositoryName" in child ): - repository_ids.append(str(child.get("repositoryId", key))) + repositories[str(child.get("repositoryId", key))] = child - if repository_ids: + if repositories: + selected_repository_id = _selected_repository_id(repositories, context) + selected_repository = repositories[selected_repository_id] return { "repository_shape": "repository-map", - "repository_ids": repository_ids, + "repository_ids": sorted(repositories), + "selected_repository_id": selected_repository_id, + "cmis_version_supported": selected_repository.get("cmisVersionSupported"), + "capabilities_present": isinstance(selected_repository.get("capabilities"), dict), + "capability_flags": _capability_flags(selected_repository), + "capability_posture": _capability_posture(selected_repository, context), } return { @@ -181,6 +223,99 @@ def _repository_facts(value: Any) -> dict[str, Any]: } +def _selected_repository_id( + repositories: dict[str, dict[str, Any]], + context: dict[str, Any], +) -> str: + configured = _opencmis_policy(context).get("repository_id") + if isinstance(configured, str) and configured in repositories: + return configured + return sorted(repositories)[0] + + +def _capability_flags(repository_info: dict[str, Any]) -> dict[str, Any]: + capabilities = repository_info.get("capabilities", {}) + return dict(capabilities) if isinstance(capabilities, dict) else {} + + +def _capability_posture( + repository_info: dict[str, Any], + context: dict[str, Any], +) -> list[dict[str, Any]]: + target = context["target_profile"] + declared = set(target.get("declared_capabilities", [])) + known_gap_refs = { + requirement_ref: gap["id"] + for gap in target.get("known_gaps", []) + for requirement_ref in gap.get("requirement_refs", []) + } + refs = sorted(declared | set(known_gap_refs)) + flags = _capability_flags(repository_info) + posture = [] + for requirement_ref in refs: + support = _requirement_support(requirement_ref, flags) + if support is True: + status = "supported" + elif support is False and requirement_ref in known_gap_refs: + status = "expected_gap" + elif support is False: + status = "unsupported" + else: + status = "unknown" + posture.append( + { + "requirement_ref": requirement_ref, + "status": status, + "known_gap_ref": known_gap_refs.get(requirement_ref), + "flag_refs": _flag_refs(requirement_ref), + } + ) + return posture + + +def _requirement_support(requirement_ref: str, flags: dict[str, Any]) -> bool | None: + if requirement_ref == "cmis.repository-info": + return True + flag_refs = _flag_refs(requirement_ref) + if not flag_refs: + return None + values = [flags[key] for key in flag_refs if key in flags] + if not values: + return None + return any(_flag_supports(value) for value in values) + + +def _flag_refs(requirement_ref: str) -> list[str]: + return { + "cmis.query": ["capabilityQuery"], + "cmis.acl": ["capabilityACL"], + "cmis.navigation-services": [ + "capabilityGetDescendants", + "capabilityGetFolderTree", + ], + "cmis.relationships": ["capabilityJoin"], + "cmis.change-log": ["capabilityChanges"], + "cmis.versioning": [ + "capabilityVersionSpecificFiling", + "capabilityPWCSearchable", + "capabilityPWCUpdatable", + ], + }.get(requirement_ref, []) + + +def _flag_supports(value: Any) -> bool: + if isinstance(value, bool): + return value + if isinstance(value, str): + return value.lower() not in {"", "false", "none", "no"} + return value is not None + + +def _opencmis_policy(context: dict[str, Any]) -> dict[str, Any]: + policy = context["assessment_profile"].get("runtime_policy", {}).get("opencmis_tck", {}) + return policy if isinstance(policy, dict) else {} + + def _write_response_artifacts( context: dict[str, Any], status_code: int, diff --git a/src/open_cmis_tck/profile.py b/src/open_cmis_tck/profile.py new file mode 100644 index 0000000..71e18cc --- /dev/null +++ b/src/open_cmis_tck/profile.py @@ -0,0 +1,146 @@ +"""CMIS-specific profile diagnostics for guide-board target profiles.""" + +from __future__ import annotations + +from typing import Any +from urllib.parse import urlparse + + +KNOWN_CMIS_REQUIREMENTS = { + "cmis.repository-info", + "cmis.type-definitions", + "cmis.object-services", + "cmis.content-streams", + "cmis.navigation-services", + "cmis.query", + "cmis.relationships", + "cmis.acl", + "cmis.policies", + "cmis.versioning", + "cmis.change-log", + "cmis.extensions", +} + + +def validate_cmis_profile_config( + target_profile: dict[str, Any], + assessment_profile: dict[str, Any] | None = None, +) -> dict[str, Any]: + """Return actionable CMIS diagnostics for guide-board profiles. + + The generic guide-board target profile remains the persisted contract. This + helper explains how the OpenCMIS extension interprets those generic fields. + """ + + diagnostics: list[dict[str, str]] = [] + browser_endpoints = [ + endpoint + for endpoint in target_profile.get("endpoints", []) + if endpoint.get("binding") == "cmis-browser" + ] + + if target_profile.get("subject_type") != "cmis-browser-binding-endpoint": + diagnostics.append( + _diagnostic( + "warning", + "subject_type", + "CMIS targets should use subject_type 'cmis-browser-binding-endpoint'.", + ) + ) + + if not browser_endpoints: + diagnostics.append( + _diagnostic( + "error", + "endpoints", + "Add one endpoint with binding 'cmis-browser' and the Browser Binding service document URL.", + ) + ) + for index, endpoint in enumerate(browser_endpoints): + parsed = urlparse(str(endpoint.get("url", ""))) + if parsed.scheme not in {"http", "https"} or not parsed.netloc: + diagnostics.append( + _diagnostic( + "error", + f"endpoints[{index}].url", + "Use an absolute http(s) CMIS Browser Binding URL.", + ) + ) + + declared = set(target_profile.get("declared_capabilities", [])) + unknown_declared = sorted(declared - KNOWN_CMIS_REQUIREMENTS) + for requirement_ref in unknown_declared: + diagnostics.append( + _diagnostic( + "warning", + "declared_capabilities", + f"Declared CMIS capability {requirement_ref!r} is not in the extension's known requirement list.", + ) + ) + + known_gap_refs = { + requirement_ref + for gap in target_profile.get("known_gaps", []) + for requirement_ref in gap.get("requirement_refs", []) + } + unexpected_gap_refs = sorted(known_gap_refs - KNOWN_CMIS_REQUIREMENTS) + for requirement_ref in unexpected_gap_refs: + diagnostics.append( + _diagnostic( + "warning", + "known_gaps", + f"Known gap {requirement_ref!r} is not in the extension's known requirement list.", + ) + ) + + runtime_policy = (assessment_profile or {}).get("runtime_policy", {}) + opencmis_policy = runtime_policy.get("opencmis_tck", {}) + if opencmis_policy and not isinstance(opencmis_policy, dict): + diagnostics.append( + _diagnostic( + "error", + "runtime_policy.opencmis_tck", + "OpenCMIS runtime policy must be an object when present.", + ) + ) + opencmis_policy = {} + + timeout = runtime_policy.get("timeout_seconds") + if assessment_profile is not None and not isinstance(timeout, (int, float)): + diagnostics.append( + _diagnostic( + "warning", + "runtime_policy.timeout_seconds", + "Set timeout_seconds to bound preflight and TCK execution.", + ) + ) + + repository_id = None + if isinstance(opencmis_policy, dict): + repository_id = opencmis_policy.get("repository_id") + if repository_id is not None and not isinstance(repository_id, str): + diagnostics.append( + _diagnostic( + "error", + "runtime_policy.opencmis_tck.repository_id", + "repository_id must be a string when configured.", + ) + ) + + status = "invalid" if any(item["severity"] == "error" for item in diagnostics) else "valid" + return { + "status": status, + "diagnostics": diagnostics, + "cmis_config": { + "browser_binding_url": browser_endpoints[0]["url"] if browser_endpoints else None, + "repository_id": repository_id, + "auth_mode": "anonymous" if target_profile.get("credentials_ref") is None else "credentials_ref", + "declared_capabilities": sorted(declared), + "known_gap_refs": sorted(known_gap_refs), + "timeout_seconds": timeout, + }, + } + + +def _diagnostic(severity: str, field: str, message: str) -> dict[str, str]: + return {"severity": severity, "field": field, "message": message} diff --git a/tests/test_open_cmis_tck.py b/tests/test_open_cmis_tck.py index 51dc228..6ad8eb1 100644 --- a/tests/test_open_cmis_tck.py +++ b/tests/test_open_cmis_tck.py @@ -1,7 +1,10 @@ from __future__ import annotations +import http.client import json +import sys import threading +import time import unittest from http.server import BaseHTTPRequestHandler, HTTPServer from pathlib import Path @@ -10,6 +13,9 @@ from tempfile import TemporaryDirectory from guide_board.discovery import discover_extensions from guide_board.execution import run_assessment from guide_board.planning import build_run_plan, validate_assessment_profile +from guide_board.retention import build_trend_summary +from guide_board.service import ServiceHandle, start_service +from open_cmis_tck.profile import validate_cmis_profile_config ROOT = Path(__file__).resolve().parents[1] @@ -44,6 +50,37 @@ class OpenCmisTckExtensionTests(unittest.TestCase): self.assertEqual(plan["extension_snapshots"][0]["path"], str(ROOT)) self.assertEqual(len(plan["ordered_steps"]), 3) + def test_validates_cmis_profile_config_with_actionable_diagnostics(self) -> None: + target = json.loads( + (ROOT / "profiles" / "targets" / "kontextual-cmis-compat.json").read_text( + encoding="utf-8" + ) + ) + assessment = json.loads( + (ROOT / "profiles" / "assessments" / "cmis-browser-baseline.json").read_text( + encoding="utf-8" + ) + ) + + diagnostics = validate_cmis_profile_config(target, assessment) + + self.assertEqual(diagnostics["status"], "valid") + self.assertEqual( + diagnostics["cmis_config"]["browser_binding_url"], + "http://127.0.0.1:8000/cmis/compat-tck/browser", + ) + self.assertEqual(diagnostics["cmis_config"]["repository_id"], "compat-tck") + self.assertEqual(diagnostics["cmis_config"]["auth_mode"], "anonymous") + + broken = dict(target) + broken["endpoints"] = [] + broken_diagnostics = validate_cmis_profile_config(broken, assessment) + self.assertEqual(broken_diagnostics["status"], "invalid") + self.assertIn( + "Add one endpoint with binding 'cmis-browser'", + broken_diagnostics["diagnostics"][0]["message"], + ) + def test_runs_cmis_preflight_against_local_endpoint(self) -> None: server = HTTPServer(("127.0.0.1", 0), _CmisHandler) thread = threading.Thread(target=server.serve_forever) @@ -88,6 +125,11 @@ class OpenCmisTckExtensionTests(unittest.TestCase): evidence[0]["facts"]["repository_ids"], ["local-test-repository"], ) + posture = { + item["requirement_ref"]: item["status"] + for item in evidence[0]["facts"]["capability_posture"] + } + self.assertEqual(posture["cmis.repository-info"], "supported") self.assertEqual(len(package["artifact_manifest"]), 2) self.assertTrue( ( @@ -103,6 +145,60 @@ class OpenCmisTckExtensionTests(unittest.TestCase): thread.join(timeout=5) server.server_close() + def test_preflight_accepts_unsupported_optional_capability_as_known_gap(self) -> None: + server = HTTPServer(("127.0.0.1", 0), _CmisHandler) + thread = threading.Thread(target=server.serve_forever) + thread.daemon = True + thread.start() + try: + with TemporaryDirectory() as temporary_directory: + temp_root = Path(temporary_directory) + target_path = temp_root / "target.json" + assessment_path = temp_root / "assessment.json" + _write_target(target_path, server.server_port, "local-cmis-query-gap") + target = json.loads(target_path.read_text(encoding="utf-8")) + target["declared_capabilities"].append("cmis.query") + target["known_gaps"].append( + { + "id": "query-not-targeted", + "requirement_refs": ["cmis.query"], + "reason": "The local fixture deliberately reports no query support.", + "status": "unsupported_by_design", + } + ) + target_path.write_text(json.dumps(target), encoding="utf-8") + _write_assessment( + assessment_path, + "local-cmis-known-gap", + "local-cmis-query-gap", + [], + None, + ) + + result = run_assessment( + CORE_ROOT, + target_path, + assessment_path, + temp_root / "run", + [ROOT], + ) + evidence = json.loads( + (Path(result["run_dir"]) / "normalized" / "evidence.json").read_text( + encoding="utf-8" + ) + )["evidence"] + posture = { + item["requirement_ref"]: item["status"] + for item in evidence[0]["facts"]["capability_posture"] + } + + self.assertEqual(result["status"], "completed") + self.assertEqual(posture["cmis.query"], "expected_gap") + finally: + server.shutdown() + thread.join(timeout=5) + server.server_close() + def test_runs_cmis_tck_command_wrapper_boundary(self) -> None: server = HTTPServer(("127.0.0.1", 0), _CmisHandler) thread = threading.Thread(target=server.serve_forever) @@ -163,6 +259,144 @@ class OpenCmisTckExtensionTests(unittest.TestCase): thread.join(timeout=5) server.server_close() + def test_runs_configured_tck_command_and_normalizes_json_results(self) -> None: + server = HTTPServer(("127.0.0.1", 0), _CmisHandler) + thread = threading.Thread(target=server.serve_forever) + thread.daemon = True + thread.start() + try: + with TemporaryDirectory() as temporary_directory: + temp_root = Path(temporary_directory) + target_path = temp_root / "target.json" + assessment_path = temp_root / "assessment.json" + fake_tck = temp_root / "fake_tck.py" + fake_tck.write_text( + "\n".join( + [ + "import json", + "print(json.dumps({", + " 'tests': [", + " {'id': 'repository-info', 'status': 'pass'},", + " {'id': 'type-definitions', 'status': 'pass'}", + " ]", + "}))", + ] + ), + encoding="utf-8", + ) + _write_target(target_path, server.server_port, "local-cmis-configured-tck") + _write_assessment( + assessment_path, + "local-cmis-configured-tck", + "local-cmis-configured-tck", + ["repository-type"], + None, + { + "requires_java_maven": False, + "repository_id": "local-test-repository", + "command": [ + sys.executable, + str(fake_tck), + "--url", + "{browser_url}", + "--repository", + "{repository_id}", + "--group", + "{check_group}", + ], + }, + ) + + result = run_assessment( + CORE_ROOT, + target_path, + assessment_path, + temp_root / "run", + [ROOT], + ) + run_dir = Path(result["run_dir"]) + evidence = json.loads( + (run_dir / "normalized" / "evidence.json").read_text(encoding="utf-8") + )["evidence"] + retention = json.loads( + (run_dir / "retention-summary.json").read_text(encoding="utf-8") + ) + trend = build_trend_summary(temp_root) + + self.assertEqual(result["status"], "completed") + self.assertEqual(evidence[1]["result"], "pass") + self.assertEqual(evidence[1]["facts"]["normalizer"], "json-cases") + self.assertEqual(evidence[1]["facts"]["result_counts"], {"pass": 2}) + self.assertTrue( + ( + run_dir + / "artifacts" + / "open-cmis-tck" + / "tck" + / "repository-type" + / "stdout.log" + ).exists() + ) + self.assertEqual(retention["summary"]["status"], "completed") + self.assertGreaterEqual(retention["summary"]["artifact_count"], 4) + self.assertEqual(trend["run_count"], 1) + finally: + server.shutdown() + thread.join(timeout=5) + server.server_close() + + def test_guide_board_service_runs_cmis_extension(self) -> None: + server = HTTPServer(("127.0.0.1", 0), _CmisHandler) + thread = threading.Thread(target=server.serve_forever) + thread.daemon = True + thread.start() + service = start_service(CORE_ROOT, [ROOT], host="127.0.0.1", port=0) + try: + with TemporaryDirectory() as temporary_directory: + temp_root = Path(temporary_directory) + target_path = temp_root / "target.json" + assessment_path = temp_root / "assessment.json" + _write_target(target_path, server.server_port, "local-cmis-service") + _write_assessment( + assessment_path, + "local-cmis-service", + "local-cmis-service", + [], + None, + ) + + extensions = _request_json(service, "GET", "/extensions") + self.assertIn( + "open-cmis-tck", + [extension["id"] for extension in extensions["extensions"]], + ) + + job = _request_json( + service, + "POST", + "/runs", + { + "target": str(target_path), + "assessment": str(assessment_path), + "output_dir": str(temp_root / "service-run"), + }, + expected_status=202, + ) + status = _wait_for_job(service, job["job_id"]) + reports = _request_json(service, "GET", f"/runs/{job['job_id']}/reports") + + self.assertEqual(status["status"], "succeeded") + self.assertEqual(status["result"]["status"], "completed") + self.assertEqual( + reports["assessment_package"]["json"]["extensions"][0]["id"], + "open-cmis-tck", + ) + finally: + service.stop() + server.shutdown() + thread.join(timeout=5) + server.server_close() + def test_preflight_failure_blocks_downstream_checks(self) -> None: with TemporaryDirectory() as temporary_directory: temp_root = Path(temporary_directory) @@ -259,7 +493,14 @@ def _write_assessment( target_id: str, check_groups: list[str], waiver_ref: str | None, + opencmis_policy: dict[str, object] | None = None, ) -> None: + runtime_policy: dict[str, object] = { + "offline": False, + "timeout_seconds": 15, + } + if opencmis_policy is not None: + runtime_policy["opencmis_tck"] = opencmis_policy path.write_text( json.dumps( { @@ -278,10 +519,7 @@ def _write_assessment( "summary_days": 365, "raw_artifact_days": 0, }, - "runtime_policy": { - "offline": False, - "timeout_seconds": 15, - }, + "runtime_policy": runtime_policy, } ), encoding="utf-8", @@ -316,6 +554,42 @@ def _write_command_waiver(path: Path, target_id: str) -> None: ) +def _request_json( + service: ServiceHandle, + method: str, + path: str, + payload: dict[str, object] | None = None, + expected_status: int = 200, +) -> dict[str, object]: + connection = http.client.HTTPConnection(service.host, service.port, timeout=5) + body = None + headers = {} + if payload is not None: + body = json.dumps(payload).encode("utf-8") + headers["Content-Type"] = "application/json" + try: + connection.request(method, path, body=body, headers=headers) + response = connection.getresponse() + data = response.read().decode("utf-8") + finally: + connection.close() + if response.status != expected_status: + raise AssertionError(f"expected HTTP {expected_status}, got {response.status}: {data}") + value = json.loads(data) + if not isinstance(value, dict): + raise AssertionError(f"expected JSON object response, got {type(value).__name__}") + return value + + +def _wait_for_job(service: ServiceHandle, job_id: str) -> dict[str, object]: + for _ in range(50): + status = _request_json(service, "GET", f"/runs/{job_id}") + if status["status"] in {"succeeded", "failed"}: + return status + time.sleep(0.05) + raise AssertionError(f"job did not finish: {job_id}") + + class _CmisHandler(BaseHTTPRequestHandler): def do_GET(self) -> None: body = json.dumps( @@ -324,7 +598,13 @@ class _CmisHandler(BaseHTTPRequestHandler): "repositoryId": "local-test-repository", "repositoryName": "Local Test Repository", "cmisVersionSupported": "1.1", - "capabilities": {}, + "capabilities": { + "capabilityACL": "discover", + "capabilityChanges": "none", + "capabilityGetDescendants": True, + "capabilityGetFolderTree": True, + "capabilityQuery": "none", + }, } } ).encode("utf-8") diff --git a/workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md b/workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md index 94f0ed1..2acf622 100644 --- a/workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md +++ b/workplans/OPEN-CMIS-TCK-WP-0001-harness-foundation.md @@ -2,10 +2,10 @@ id: OPEN-CMIS-TCK-WP-0001 type: extension-workplan title: "OpenCMIS TCK Harness Foundation" -repo: guide-board +repo: open-cmis-tck extension: open-cmis-tck domain: markitect -status: active +status: completed owner: codex planning_priority: high planning_order: 2 @@ -81,7 +81,7 @@ Progress: ```task id: OPEN-CMIS-TCK-WP-0001-T002 -status: todo +status: done priority: high state_hub_task_id: "2ccc74a7-bed9-4769-8608-d579fdf3a0cd" ``` @@ -95,11 +95,19 @@ Acceptance: - Profile validation produces actionable diagnostics for missing or invalid fields. +Progress: + +- Added CMIS-specific profile diagnostics in `open_cmis_tck.profile`. +- Documented the CMIS profile contract in `docs/CMIS-PROFILES.md`. +- Added `runtime_policy.opencmis_tck.repository_id` to the baseline assessment + profile while keeping the persisted profiles compatible with guide-board core + schemas. + ## D1.3 - CMIS Preflight Probe ```task id: OPEN-CMIS-TCK-WP-0001-T003 -status: in_progress +status: done priority: high state_hub_task_id: "6d45885b-78a4-4e8b-8fcc-b8d6488e703b" ``` @@ -119,14 +127,15 @@ Progress: assessment-package fingerprinting. - Failed CMIS preflight now blocks downstream OpenCMIS TCK groups instead of invoking the Java/Maven wrapper against an invalid target. -- Capability flag normalization remains to be expanded after a live target sample - is captured. +- Capability flags are normalized into repository capability posture facts. +- Unsupported optional capabilities can be accepted as target-profile known gaps + without hiding unexpected unsupported declared capabilities. ## D1.4 - OpenCMIS TCK Runner Wrapper ```task id: OPEN-CMIS-TCK-WP-0001-T004 -status: in_progress +status: done priority: high state_hub_task_id: "502d7586-6f9e-475e-9683-43260666d5d9" ``` @@ -145,14 +154,17 @@ Progress: evidence when dependencies or final TCK invocation details are missing. - `profiles/expectations/cmis-local-harness.json` marks local bootstrap blockers as expected without hiding target preflight failures. -- Actual Apache Chemistry TCK classpath resolution, group invocation, and raw log - capture remain to be implemented. +- The wrapper can invoke a configured argv command from + `runtime_policy.opencmis_tck.command` or environment variables. +- The wrapper expands CMIS placeholders, captures stdout/stderr and invocation + metadata under the run directory, and skips cleanly when Java/Maven or command + configuration are missing. ## D1.5 - CMIS Result Normalization ```task id: OPEN-CMIS-TCK-WP-0001-T005 -status: todo +status: done priority: high state_hub_task_id: "716486b6-6f14-41f8-8417-5015ba746005" ``` @@ -165,11 +177,19 @@ Acceptance: - Failures include enough context to map back to TCK group, capability group, target profile, and raw artifact paths. +Progress: + +- Added runner-side normalization for JSON case lists, JSON runner results, + JUnit-style XML, and exit-code-only output. +- Normalized result counts, case IDs, selected check group, target URL, + repository ID, return code, and artifact paths are captured in guide-board + evidence facts. + ## D1.6 - Capability Mapping And Reports ```task id: OPEN-CMIS-TCK-WP-0001-T006 -status: in_progress +status: done priority: high state_hub_task_id: "9f7dacc5-4d19-4755-aa9a-8572d4285514" ``` @@ -190,12 +210,16 @@ Progress: groups. - Guide-board writes normalized mapping records and includes capability-group counts in Markdown reports. +- Mapping coverage now includes relationships, change log, policy, and extension + gap requirement refs. +- Additional check groups are declared for relationships, change log, and + extension/known-gap review. ## D1.7 - Optional Local Service API Adapter ```task id: OPEN-CMIS-TCK-WP-0001-T007 -status: todo +status: done priority: medium state_hub_task_id: "a05e47bd-88db-4878-aef4-bf328790c3f0" ``` @@ -207,11 +231,17 @@ Acceptance: - CLI operation remains the primary path. - Long-running TCK jobs are tracked without blocking the API process. +Progress: + +- Verified `open-cmis-tck` through guide-board's local service as an external + extension. +- Documented service usage in `docs/SERVICE-AND-RETENTION.md`. + ## D1.8 - Historical Result Retention ```task id: OPEN-CMIS-TCK-WP-0001-T008 -status: todo +status: done priority: medium state_hub_task_id: "c27ea43f-41ec-49d0-a890-3681455f7c6c" ``` @@ -223,6 +253,14 @@ Acceptance: - Summaries are suitable for trend charts and downstream capability-score updates. +Progress: + +- CMIS runs now exercise guide-board retention summaries and trend input through + extension tests. +- The sample assessment profile declares 365-day summary retention and 30-day + raw artifact retention. +- Retention behavior is documented in `docs/SERVICE-AND-RETENTION.md`. + ## Definition Of Done - A developer can configure a CMIS Browser Binding endpoint.