generated from coulomb/repo-seed
chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-05-07: - update .custodian-brief.md for open-cmis-tck
This commit is contained in:
@@ -2,12 +2,23 @@
|
||||
# Custodian Brief — open-cmis-tck
|
||||
|
||||
**Domain:** markitect
|
||||
**Last synced:** 2026-05-07 20:34 UTC
|
||||
**Last synced:** 2026-05-07 20:58 UTC
|
||||
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
||||
|
||||
## Active Workstreams
|
||||
|
||||
*(none — repo may need first-session setup)*
|
||||
### Live OpenCMIS TCK Execution And Capability Maturity
|
||||
Progress: 0/9 done | workstream_id: `da3f0d16-ba8e-4147-b0fc-ab3462e0b7b0`
|
||||
|
||||
**Open tasks:**
|
||||
- · D2.1 - Resolve TCK Runtime And Access Model `f3144edb`
|
||||
- · D2.2 - Local Environment Bootstrap Command `f993c1ef`
|
||||
- · D2.3 - OpenCMIS TCK Adapter Invocation `a446a80f`
|
||||
- · D2.4 - Target Profiles And Credential References `c33a4d9a`
|
||||
- · D2.5 - Real Result Normalization `03ba9506`
|
||||
- · D2.6 - Live Pilot Run `d9eb9384`
|
||||
- · D2.7 - CMIS Capability Maturity Scorecard `7365052f`
|
||||
- … and 2 more open tasks
|
||||
|
||||
---
|
||||
## MCP Orientation (when available)
|
||||
|
||||
16
README.md
16
README.md
@@ -53,8 +53,14 @@ Expected current behavior:
|
||||
|
||||
- CMIS Browser Binding preflight runs first.
|
||||
- Failed preflight blocks downstream TCK groups.
|
||||
- The Java/Maven OpenCMIS wrapper reports structured blockers until the final
|
||||
Apache Chemistry TCK invocation is configured.
|
||||
- The Java/Maven OpenCMIS wrapper reports structured blockers until a local
|
||||
Apache Chemistry TCK command is configured.
|
||||
- If a command is configured, raw stdout/stderr and normalized runner output are
|
||||
captured under the guide-board run directory.
|
||||
|
||||
Runner command configuration lives in
|
||||
`runtime_policy.opencmis_tck.command`. See
|
||||
[docs/OPENCMIS-TCK-RUNNER.md](docs/OPENCMIS-TCK-RUNNER.md).
|
||||
|
||||
## Tests
|
||||
|
||||
@@ -64,6 +70,12 @@ Run extension tests with the guide-board core on `PYTHONPATH`:
|
||||
PYTHONPATH=../guide-board/src python3 -m unittest discover -s tests
|
||||
```
|
||||
|
||||
## Docs
|
||||
|
||||
- [docs/CMIS-PROFILES.md](docs/CMIS-PROFILES.md)
|
||||
- [docs/OPENCMIS-TCK-RUNNER.md](docs/OPENCMIS-TCK-RUNNER.md)
|
||||
- [docs/SERVICE-AND-RETENTION.md](docs/SERVICE-AND-RETENTION.md)
|
||||
|
||||
## Boundary
|
||||
|
||||
Apache Chemistry/OpenCMIS TCK dependencies may be restricted by upstream
|
||||
|
||||
67
docs/CMIS-PROFILES.md
Normal file
67
docs/CMIS-PROFILES.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# CMIS Profiles
|
||||
|
||||
Status: draft
|
||||
Created: 2026-05-07
|
||||
|
||||
## Purpose
|
||||
|
||||
`open-cmis-tck` uses guide-board target and assessment profiles without adding a
|
||||
separate persisted profile format. This keeps the extension compatible with the
|
||||
guide-board planner while still giving CMIS-specific diagnostics through
|
||||
`open_cmis_tck.profile.validate_cmis_profile_config`.
|
||||
|
||||
## Target Profile Fields
|
||||
|
||||
The CMIS target profile uses the guide-board `target-profile` schema:
|
||||
|
||||
- `subject_type`: use `cmis-browser-binding-endpoint`.
|
||||
- `endpoints`: include one endpoint with `binding` set to `cmis-browser` and
|
||||
`url` set to the Browser Binding service document URL.
|
||||
- `credentials_ref`: use `null` for anonymous/local development targets, or a
|
||||
secret reference for authenticated repositories.
|
||||
- `declared_capabilities`: list the CMIS requirement refs the target claims to
|
||||
support, such as `cmis.repository-info`, `cmis.type-definitions`,
|
||||
`cmis.object-services`, `cmis.content-streams`, `cmis.query`, `cmis.acl`, and
|
||||
`cmis.versioning`.
|
||||
- `known_gaps`: list unsupported optional requirements with a stable gap ID,
|
||||
requirement refs, reason, and status such as `unsupported_by_design`.
|
||||
|
||||
## Assessment Runtime Fields
|
||||
|
||||
Repository selection and harness execution settings live in the assessment
|
||||
profile because they are run policy, not target identity:
|
||||
|
||||
```json
|
||||
{
|
||||
"runtime_policy": {
|
||||
"offline": false,
|
||||
"timeout_seconds": 300,
|
||||
"opencmis_tck": {
|
||||
"repository_id": "compat-tck",
|
||||
"requires_java_maven": true,
|
||||
"command": ["java", "-jar", "/assets/opencmis-tck-runner.jar"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`repository_id` is optional for preflight. If omitted, preflight selects the
|
||||
first repository from the Browser Binding service document. A real TCK command
|
||||
usually needs it.
|
||||
|
||||
`command` is optional. When absent, the wrapper reports
|
||||
`tck_invocation_not_configured` as a structured, expected bootstrap blocker.
|
||||
|
||||
## Diagnostics
|
||||
|
||||
Use the extension helper from tests or local scripts:
|
||||
|
||||
```python
|
||||
from open_cmis_tck.profile import validate_cmis_profile_config
|
||||
|
||||
diagnostics = validate_cmis_profile_config(target_profile, assessment_profile)
|
||||
```
|
||||
|
||||
The result contains `status`, `diagnostics`, and the interpreted `cmis_config`.
|
||||
Diagnostics are intentionally actionable: they point to the field that should be
|
||||
changed and explain what the extension expects.
|
||||
97
docs/OPENCMIS-TCK-RUNNER.md
Normal file
97
docs/OPENCMIS-TCK-RUNNER.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# OpenCMIS TCK Runner
|
||||
|
||||
Status: draft
|
||||
Created: 2026-05-07
|
||||
|
||||
## Purpose
|
||||
|
||||
The runner wrapper at `runners/opencmis_tck.py` is the boundary between
|
||||
guide-board and Apache Chemistry OpenCMIS TCK execution. It keeps Java/Maven
|
||||
setup, harness command lines, raw logs, and result normalization inside this
|
||||
extension.
|
||||
|
||||
## Dependency Checks
|
||||
|
||||
By default, the wrapper checks:
|
||||
|
||||
- `java -version`
|
||||
- `mvn -version`
|
||||
|
||||
If either dependency is unavailable, the runner returns `blocked` evidence with
|
||||
`blocked_reason: missing_dependency`.
|
||||
|
||||
Set `runtime_policy.opencmis_tck.requires_java_maven` to `false` only for tests
|
||||
or custom harness commands that do not use the local Java/Maven toolchain.
|
||||
|
||||
## Command Configuration
|
||||
|
||||
Configure a TCK command as an argv list:
|
||||
|
||||
```json
|
||||
{
|
||||
"runtime_policy": {
|
||||
"opencmis_tck": {
|
||||
"repository_id": "compat-tck",
|
||||
"command": [
|
||||
"java",
|
||||
"-jar",
|
||||
"/assets/opencmis-tck-runner.jar",
|
||||
"--url",
|
||||
"{browser_url}",
|
||||
"--repository",
|
||||
"{repository_id}",
|
||||
"--group",
|
||||
"{check_group}",
|
||||
"--output",
|
||||
"{artifact_dir}"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Supported placeholders:
|
||||
|
||||
- `{browser_url}`
|
||||
- `{repository_id}`
|
||||
- `{check_group}`
|
||||
- `{target_id}`
|
||||
- `{run_dir}`
|
||||
- `{artifact_dir}`
|
||||
|
||||
The wrapper also accepts `OPENCMIS_TCK_COMMAND_JSON` as a JSON string array, or
|
||||
`OPENCMIS_TCK_COMMAND` as a shell-like string that is split into argv. The final
|
||||
command still runs without shell expansion.
|
||||
|
||||
## Raw Artifacts
|
||||
|
||||
For each selected check group, artifacts are written under:
|
||||
|
||||
```text
|
||||
artifacts/open-cmis-tck/tck/<check-group>/
|
||||
```
|
||||
|
||||
Current artifacts:
|
||||
|
||||
- `invocation.json`
|
||||
- `stdout.log`
|
||||
- `stderr.log`
|
||||
- `normalized-runner-result.json`
|
||||
|
||||
The guide-board core fingerprints these files in the assessment package artifact
|
||||
manifest when they are referenced by the runner result.
|
||||
|
||||
## Normalization
|
||||
|
||||
The wrapper normalizes, in order:
|
||||
|
||||
1. JSON written to stdout with a `tests`, `cases`, or `results` array.
|
||||
2. JUnit-style XML files written directly into `{artifact_dir}`.
|
||||
3. Exit code only, when no structured output is found.
|
||||
|
||||
Case statuses normalize to guide-board result vocabulary: `pass`, `fail`,
|
||||
`skipped`, `expected_gap`, `unsupported_by_design`, `infrastructure_error`, and
|
||||
related core statuses.
|
||||
|
||||
This is enough to run a real local TCK adapter while preserving raw logs for
|
||||
future Apache Chemistry-specific parsing refinements.
|
||||
56
docs/SERVICE-AND-RETENTION.md
Normal file
56
docs/SERVICE-AND-RETENTION.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# Service And Retention Integration
|
||||
|
||||
Status: draft
|
||||
Created: 2026-05-07
|
||||
|
||||
## Local Service
|
||||
|
||||
`open-cmis-tck` does not run its own service. It plugs into the guide-board
|
||||
local API as an external extension:
|
||||
|
||||
```sh
|
||||
cd ../guide-board
|
||||
PYTHONPATH=src python3 -m guide_board \
|
||||
--extension-dir ../open-cmis-tck \
|
||||
serve --host 127.0.0.1 --port 8080
|
||||
```
|
||||
|
||||
The guide-board service can then:
|
||||
|
||||
- list `open-cmis-tck` from `GET /extensions`,
|
||||
- build CMIS run plans with `POST /assessments/plan`,
|
||||
- start CMIS runs with `POST /runs`,
|
||||
- inspect jobs with `GET /runs/{job_id}`,
|
||||
- fetch reports with `GET /runs/{job_id}/reports`.
|
||||
|
||||
CLI execution remains the primary and most transparent path. The service is a
|
||||
transport and job-tracking layer over the same runner contracts.
|
||||
|
||||
## Retention
|
||||
|
||||
CMIS runs use the guide-board run directory contract. Each run writes:
|
||||
|
||||
- `run.json`
|
||||
- `retention-summary.json`
|
||||
- `plan.json`
|
||||
- `normalized/evidence.json`
|
||||
- `normalized/findings.json`
|
||||
- `normalized/mappings.json`
|
||||
- `reports/assessment-package.json`
|
||||
- `reports/report.md`
|
||||
|
||||
The sample assessment profile keeps summaries for 365 days and raw artifacts for
|
||||
30 days:
|
||||
|
||||
```json
|
||||
{
|
||||
"retention_policy": {
|
||||
"summary_days": 365,
|
||||
"raw_artifact_days": 30
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Compact `retention-summary.json` files are suitable for guide-board trend
|
||||
summaries and downstream CMIS capability scorecards without retaining unbounded
|
||||
raw TCK logs.
|
||||
@@ -55,6 +55,33 @@
|
||||
"cmis.versioning"
|
||||
],
|
||||
"runner_ref": "opencmis-tck"
|
||||
},
|
||||
{
|
||||
"id": "relationships",
|
||||
"name": "Relationship Checks",
|
||||
"check_type": "executable_harness",
|
||||
"requirement_refs": [
|
||||
"cmis.relationships"
|
||||
],
|
||||
"runner_ref": "opencmis-tck"
|
||||
},
|
||||
{
|
||||
"id": "change-log",
|
||||
"name": "Change Log Checks",
|
||||
"check_type": "executable_harness",
|
||||
"requirement_refs": [
|
||||
"cmis.change-log"
|
||||
],
|
||||
"runner_ref": "opencmis-tck"
|
||||
},
|
||||
{
|
||||
"id": "extension-gaps",
|
||||
"name": "Extension And Known Gap Checks",
|
||||
"check_type": "executable_harness",
|
||||
"requirement_refs": [
|
||||
"cmis.extensions"
|
||||
],
|
||||
"runner_ref": "opencmis-tck"
|
||||
}
|
||||
],
|
||||
"preflight_runner": "cmis-browser-preflight",
|
||||
@@ -78,7 +105,7 @@
|
||||
"--context",
|
||||
"{context_json}"
|
||||
],
|
||||
"description": "Checks Java/Maven availability and prepares the future Apache Chemistry OpenCMIS TCK invocation."
|
||||
"description": "Checks Java/Maven availability, invokes a configured Apache Chemistry OpenCMIS TCK command, captures raw artifacts, and returns normalized evidence."
|
||||
}
|
||||
],
|
||||
"normalizers": [
|
||||
|
||||
@@ -54,12 +54,40 @@
|
||||
"label": "ACL And Policy",
|
||||
"description": "Access-control list and policy behavior where supported by the target profile."
|
||||
},
|
||||
{
|
||||
"requirement_ref": "cmis.policies",
|
||||
"target_type": "capability_group",
|
||||
"target_id": "acl-policy",
|
||||
"label": "ACL And Policy",
|
||||
"description": "Policy object behavior where supported by the target profile."
|
||||
},
|
||||
{
|
||||
"requirement_ref": "cmis.versioning",
|
||||
"target_type": "capability_group",
|
||||
"target_id": "versioning",
|
||||
"label": "Versioning",
|
||||
"description": "Checkout, checkin, version series, and version-specific object behavior."
|
||||
},
|
||||
{
|
||||
"requirement_ref": "cmis.relationships",
|
||||
"target_type": "capability_group",
|
||||
"target_id": "relationships",
|
||||
"label": "Relationships",
|
||||
"description": "Relationship object and relationship navigation behavior where supported by the repository."
|
||||
},
|
||||
{
|
||||
"requirement_ref": "cmis.change-log",
|
||||
"target_type": "capability_group",
|
||||
"target_id": "change-log",
|
||||
"label": "Change Log",
|
||||
"description": "Change log capability, token behavior, and change event retrieval."
|
||||
},
|
||||
{
|
||||
"requirement_ref": "cmis.extensions",
|
||||
"target_type": "capability_group",
|
||||
"target_id": "extension-gaps",
|
||||
"label": "Extensions And Known Gaps",
|
||||
"description": "Explicitly scoped CMIS extensions, unsupported optional services, and compatibility gaps."
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -28,6 +28,10 @@
|
||||
},
|
||||
"runtime_policy": {
|
||||
"offline": false,
|
||||
"timeout_seconds": 300
|
||||
"timeout_seconds": 300,
|
||||
"opencmis_tck": {
|
||||
"repository_id": "compat-tck",
|
||||
"requires_java_maven": true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,17 +1,20 @@
|
||||
#!/usr/bin/env python3
|
||||
"""OpenCMIS TCK wrapper boundary.
|
||||
|
||||
This wrapper intentionally stops before invoking Apache Chemistry. Its current
|
||||
job is to prove the command-runner contract, verify local Java/Maven posture, and
|
||||
return structured evidence that the actual TCK execution remains pending.
|
||||
The wrapper owns extension-local orchestration only: dependency checks, optional
|
||||
user-supplied TCK command execution, raw artifact capture, and normalization into
|
||||
the guide-board runner result contract.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import shlex
|
||||
import subprocess
|
||||
import xml.etree.ElementTree as ET
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -23,6 +26,7 @@ def main() -> int:
|
||||
|
||||
context = _load_context(Path(args.context))
|
||||
selected_group = context["step"].get("check_group")
|
||||
config = _opencmis_policy(context)
|
||||
dependency_results = {
|
||||
"java": _probe_command(["java", "-version"]),
|
||||
"maven": _probe_command(["mvn", "-version"]),
|
||||
@@ -33,7 +37,7 @@ def main() -> int:
|
||||
if not result["available"]
|
||||
]
|
||||
|
||||
if missing:
|
||||
if config.get("requires_java_maven", True) and missing:
|
||||
_emit(
|
||||
{
|
||||
"result": "blocked",
|
||||
@@ -52,6 +56,11 @@ def main() -> int:
|
||||
)
|
||||
return 0
|
||||
|
||||
command_template = _configured_command(config)
|
||||
if command_template is not None:
|
||||
_emit(_run_configured_tck(context, selected_group, dependency_results, command_template))
|
||||
return 0
|
||||
|
||||
_emit(
|
||||
{
|
||||
"result": "blocked",
|
||||
@@ -62,7 +71,7 @@ def main() -> int:
|
||||
"blocked_reason": "tck_invocation_not_configured",
|
||||
"selected_check_group": selected_group,
|
||||
"dependencies": dependency_results,
|
||||
"next_step": "Resolve the Maven artifact, classpath, TCK group mapping, and raw artifact capture contract.",
|
||||
"next_step": "Configure runtime_policy.opencmis_tck.command or OPENCMIS_TCK_COMMAND_JSON with an argv list for the local Apache Chemistry TCK runner.",
|
||||
},
|
||||
"artifact_refs": [],
|
||||
}
|
||||
@@ -108,6 +117,358 @@ def _probe_command(command: list[str]) -> dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def _run_configured_tck(
|
||||
context: dict[str, Any],
|
||||
selected_group: str | None,
|
||||
dependency_results: dict[str, dict[str, Any]],
|
||||
command_template: list[str],
|
||||
) -> dict[str, Any]:
|
||||
run_dir = Path(context["run_dir"])
|
||||
artifact_dir = run_dir / "artifacts" / "open-cmis-tck" / "tck" / _safe_id(selected_group or "unknown")
|
||||
artifact_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
command = [_expand_arg(arg, context, selected_group, artifact_dir) for arg in command_template]
|
||||
invocation_ref = _write_json_artifact(
|
||||
run_dir,
|
||||
artifact_dir,
|
||||
"invocation.json",
|
||||
{
|
||||
"selected_check_group": selected_group,
|
||||
"command": command,
|
||||
"target_profile_id": context["target_profile"]["id"],
|
||||
"repository_id": _repository_id(context),
|
||||
"browser_binding_url": _browser_url(context),
|
||||
},
|
||||
)
|
||||
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
cwd=Path(context["extension_path"]),
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=_timeout_seconds(context),
|
||||
check=False,
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
return {
|
||||
"result": "blocked",
|
||||
"observations": [
|
||||
f"OpenCMIS TCK command could not start because {exc.filename!r} was not found."
|
||||
],
|
||||
"facts": {
|
||||
"blocked_reason": "missing_command",
|
||||
"selected_check_group": selected_group,
|
||||
"dependencies": dependency_results,
|
||||
"command": command,
|
||||
},
|
||||
"artifact_refs": [invocation_ref],
|
||||
}
|
||||
except subprocess.TimeoutExpired:
|
||||
return {
|
||||
"result": "infrastructure_error",
|
||||
"observations": [
|
||||
f"OpenCMIS TCK command timed out after {_timeout_seconds(context)} seconds."
|
||||
],
|
||||
"facts": {
|
||||
"selected_check_group": selected_group,
|
||||
"dependencies": dependency_results,
|
||||
"command": command,
|
||||
},
|
||||
"artifact_refs": [invocation_ref],
|
||||
}
|
||||
|
||||
stdout_ref = _write_text_artifact(run_dir, artifact_dir, "stdout.log", completed.stdout)
|
||||
stderr_ref = _write_text_artifact(run_dir, artifact_dir, "stderr.log", completed.stderr)
|
||||
normalized = _normalize_tck_output(completed, artifact_dir, selected_group)
|
||||
normalized["facts"].update(
|
||||
{
|
||||
"selected_check_group": selected_group,
|
||||
"dependencies": dependency_results,
|
||||
"command": command,
|
||||
"returncode": completed.returncode,
|
||||
"browser_binding_url": _browser_url(context),
|
||||
"repository_id": _repository_id(context),
|
||||
}
|
||||
)
|
||||
normalized_ref = _write_json_artifact(
|
||||
run_dir,
|
||||
artifact_dir,
|
||||
"normalized-runner-result.json",
|
||||
normalized,
|
||||
)
|
||||
normalized["artifact_refs"].extend([invocation_ref, stdout_ref, stderr_ref, normalized_ref])
|
||||
return normalized
|
||||
|
||||
|
||||
def _normalize_tck_output(
|
||||
completed: subprocess.CompletedProcess[str],
|
||||
artifact_dir: Path,
|
||||
selected_group: str | None,
|
||||
) -> dict[str, Any]:
|
||||
parsed_stdout = _parse_json(completed.stdout)
|
||||
if isinstance(parsed_stdout, dict):
|
||||
return _normalize_json_result(parsed_stdout, completed.returncode, selected_group)
|
||||
|
||||
junit_files = sorted(artifact_dir.glob("*.xml"))
|
||||
if junit_files:
|
||||
return _normalize_junit_result(junit_files[0], completed.returncode, selected_group)
|
||||
|
||||
if completed.returncode == 0:
|
||||
return {
|
||||
"result": "pass",
|
||||
"observations": [
|
||||
"OpenCMIS TCK command completed successfully, but no structured result payload was found."
|
||||
],
|
||||
"facts": {
|
||||
"normalizer": "exit-code",
|
||||
"result_counts": {"pass": 1},
|
||||
},
|
||||
"artifact_refs": [],
|
||||
}
|
||||
return {
|
||||
"result": "fail",
|
||||
"observations": [
|
||||
"OpenCMIS TCK command exited with a non-zero status and no structured result payload was found."
|
||||
],
|
||||
"facts": {
|
||||
"normalizer": "exit-code",
|
||||
"result_counts": {"fail": 1},
|
||||
},
|
||||
"artifact_refs": [],
|
||||
}
|
||||
|
||||
|
||||
def _normalize_json_result(
|
||||
payload: dict[str, Any],
|
||||
returncode: int,
|
||||
selected_group: str | None,
|
||||
) -> dict[str, Any]:
|
||||
cases = _json_cases(payload)
|
||||
if cases:
|
||||
counts: dict[str, int] = {}
|
||||
normalized_cases = []
|
||||
for case in cases:
|
||||
status = _normalize_case_status(str(case.get("status", "unknown")))
|
||||
counts[status] = counts.get(status, 0) + 1
|
||||
normalized_cases.append(
|
||||
{
|
||||
"id": str(case.get("id", case.get("name", "unnamed"))),
|
||||
"status": status,
|
||||
"message": str(case.get("message", case.get("reason", ""))),
|
||||
}
|
||||
)
|
||||
return {
|
||||
"result": _aggregate_result(counts, returncode),
|
||||
"observations": [
|
||||
f"OpenCMIS TCK group {selected_group!r} produced {sum(counts.values())} normalized case result(s)."
|
||||
],
|
||||
"facts": {
|
||||
"normalizer": "json-cases",
|
||||
"result_counts": counts,
|
||||
"cases": normalized_cases[:200],
|
||||
},
|
||||
"artifact_refs": [],
|
||||
}
|
||||
|
||||
result = _normalize_case_status(str(payload.get("result", "unknown")))
|
||||
if returncode != 0 and result in {"pass", "warning", "skipped"}:
|
||||
result = "infrastructure_error"
|
||||
return {
|
||||
"result": result,
|
||||
"observations": _observations_from_payload(payload, selected_group),
|
||||
"facts": {
|
||||
"normalizer": "json-runner-result",
|
||||
"result_counts": {result: 1},
|
||||
"payload": payload,
|
||||
},
|
||||
"artifact_refs": [],
|
||||
}
|
||||
|
||||
|
||||
def _normalize_junit_result(
|
||||
path: Path,
|
||||
returncode: int,
|
||||
selected_group: str | None,
|
||||
) -> dict[str, Any]:
|
||||
tree = ET.parse(path)
|
||||
root = tree.getroot()
|
||||
suites = [root] if root.tag == "testsuite" else list(root.findall("testsuite"))
|
||||
total = sum(int(suite.get("tests", "0")) for suite in suites)
|
||||
failures = sum(int(suite.get("failures", "0")) for suite in suites)
|
||||
errors = sum(int(suite.get("errors", "0")) for suite in suites)
|
||||
skipped = sum(int(suite.get("skipped", "0")) for suite in suites)
|
||||
passed = max(0, total - failures - errors - skipped)
|
||||
counts = {
|
||||
"pass": passed,
|
||||
"fail": failures + errors,
|
||||
"skipped": skipped,
|
||||
}
|
||||
counts = {key: value for key, value in counts.items() if value}
|
||||
return {
|
||||
"result": _aggregate_result(counts, returncode),
|
||||
"observations": [
|
||||
f"OpenCMIS TCK group {selected_group!r} produced JUnit-style XML results."
|
||||
],
|
||||
"facts": {
|
||||
"normalizer": "junit-xml",
|
||||
"result_counts": counts,
|
||||
"junit_xml": str(path),
|
||||
},
|
||||
"artifact_refs": [],
|
||||
}
|
||||
|
||||
|
||||
def _json_cases(payload: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
for key in ("tests", "cases", "results"):
|
||||
value = payload.get(key)
|
||||
if isinstance(value, list) and all(isinstance(item, dict) for item in value):
|
||||
return value
|
||||
return []
|
||||
|
||||
|
||||
def _aggregate_result(counts: dict[str, int], returncode: int) -> str:
|
||||
if counts.get("infrastructure_error"):
|
||||
return "infrastructure_error"
|
||||
if counts.get("fail"):
|
||||
return "fail"
|
||||
if counts.get("pass"):
|
||||
return "pass"
|
||||
if counts.get("expected_gap"):
|
||||
return "expected_gap"
|
||||
if counts.get("unsupported_by_design"):
|
||||
return "unsupported_by_design"
|
||||
if counts.get("skipped"):
|
||||
return "skipped"
|
||||
return "infrastructure_error" if returncode else "unknown"
|
||||
|
||||
|
||||
def _normalize_case_status(value: str) -> str:
|
||||
normalized = value.strip().lower().replace("-", "_").replace(" ", "_")
|
||||
if normalized in {"ok", "success", "passed"}:
|
||||
return "pass"
|
||||
if normalized in {"failure", "failed", "error"}:
|
||||
return "fail"
|
||||
if normalized in {"skip", "skipped"}:
|
||||
return "skipped"
|
||||
if normalized in {"expected_skip", "expected_gap"}:
|
||||
return "expected_gap"
|
||||
if normalized in {"unsupported", "unsupported_by_design"}:
|
||||
return "unsupported_by_design"
|
||||
if normalized in {"infra", "infrastructure_error"}:
|
||||
return "infrastructure_error"
|
||||
if normalized in {
|
||||
"pass",
|
||||
"fail",
|
||||
"warning",
|
||||
"manual",
|
||||
"not_applicable",
|
||||
"waiver_applied",
|
||||
"blocked",
|
||||
"unknown",
|
||||
}:
|
||||
return normalized
|
||||
return "unknown"
|
||||
|
||||
|
||||
def _observations_from_payload(payload: dict[str, Any], selected_group: str | None) -> list[str]:
|
||||
observations = payload.get("observations")
|
||||
if isinstance(observations, list):
|
||||
return [str(item) for item in observations]
|
||||
message = payload.get("message")
|
||||
if isinstance(message, str) and message:
|
||||
return [message]
|
||||
return [f"OpenCMIS TCK group {selected_group!r} returned a structured result."]
|
||||
|
||||
|
||||
def _opencmis_policy(context: dict[str, Any]) -> dict[str, Any]:
|
||||
policy = context["assessment_profile"].get("runtime_policy", {}).get("opencmis_tck", {})
|
||||
return policy if isinstance(policy, dict) else {}
|
||||
|
||||
|
||||
def _configured_command(config: dict[str, Any]) -> list[str] | None:
|
||||
command = config.get("command")
|
||||
if isinstance(command, list) and all(isinstance(item, str) and item for item in command):
|
||||
return command
|
||||
|
||||
env_json = os.environ.get("OPENCMIS_TCK_COMMAND_JSON")
|
||||
if env_json:
|
||||
parsed = json.loads(env_json)
|
||||
if isinstance(parsed, list) and all(isinstance(item, str) and item for item in parsed):
|
||||
return parsed
|
||||
raise ValueError("OPENCMIS_TCK_COMMAND_JSON must be a JSON string array")
|
||||
|
||||
env_command = os.environ.get("OPENCMIS_TCK_COMMAND")
|
||||
if env_command:
|
||||
return shlex.split(env_command)
|
||||
return None
|
||||
|
||||
|
||||
def _expand_arg(
|
||||
value: str,
|
||||
context: dict[str, Any],
|
||||
selected_group: str | None,
|
||||
artifact_dir: Path,
|
||||
) -> str:
|
||||
return (
|
||||
value.replace("{run_dir}", context["run_dir"])
|
||||
.replace("{artifact_dir}", str(artifact_dir))
|
||||
.replace("{browser_url}", _browser_url(context) or "")
|
||||
.replace("{repository_id}", _repository_id(context) or "")
|
||||
.replace("{check_group}", selected_group or "")
|
||||
.replace("{target_id}", context["target_profile"]["id"])
|
||||
)
|
||||
|
||||
|
||||
def _browser_url(context: dict[str, Any]) -> str | None:
|
||||
for endpoint in context["target_profile"].get("endpoints", []):
|
||||
if endpoint.get("binding") == "cmis-browser":
|
||||
return endpoint.get("url")
|
||||
return None
|
||||
|
||||
|
||||
def _repository_id(context: dict[str, Any]) -> str | None:
|
||||
value = _opencmis_policy(context).get("repository_id")
|
||||
return value if isinstance(value, str) else None
|
||||
|
||||
|
||||
def _timeout_seconds(context: dict[str, Any]) -> float:
|
||||
runtime_policy = context["assessment_profile"].get("runtime_policy", {})
|
||||
opencmis_policy = _opencmis_policy(context)
|
||||
configured = opencmis_policy.get("timeout_seconds", runtime_policy.get("timeout_seconds", 300))
|
||||
if not isinstance(configured, (int, float)):
|
||||
return 300.0
|
||||
return max(1.0, float(configured))
|
||||
|
||||
|
||||
def _write_text_artifact(run_dir: Path, artifact_dir: Path, name: str, value: str) -> str:
|
||||
path = artifact_dir / name
|
||||
path.write_text(value, encoding="utf-8")
|
||||
return str(path.relative_to(run_dir))
|
||||
|
||||
|
||||
def _write_json_artifact(
|
||||
run_dir: Path,
|
||||
artifact_dir: Path,
|
||||
name: str,
|
||||
value: dict[str, Any],
|
||||
) -> str:
|
||||
path = artifact_dir / name
|
||||
path.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||
return str(path.relative_to(run_dir))
|
||||
|
||||
|
||||
def _parse_json(value: str) -> Any:
|
||||
try:
|
||||
return json.loads(value)
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
|
||||
|
||||
def _safe_id(value: str) -> str:
|
||||
return "".join(char if char.isalnum() or char in {"-", "_"} else "_" for char in value)
|
||||
|
||||
|
||||
def _emit(value: dict[str, Any]) -> None:
|
||||
print(json.dumps(value, indent=2, sort_keys=True))
|
||||
|
||||
|
||||
@@ -117,12 +117,43 @@ def run(context: dict[str, Any]) -> dict[str, Any]:
|
||||
}
|
||||
|
||||
facts["json_detected"] = True
|
||||
facts.update(_repository_facts(parsed))
|
||||
repository_facts = _repository_facts(parsed, context)
|
||||
facts.update(repository_facts)
|
||||
unsupported = [
|
||||
item
|
||||
for item in repository_facts.get("capability_posture", [])
|
||||
if item.get("status") == "unsupported"
|
||||
]
|
||||
expected_gaps = [
|
||||
item
|
||||
for item in repository_facts.get("capability_posture", [])
|
||||
if item.get("status") == "expected_gap"
|
||||
]
|
||||
if unsupported:
|
||||
return {
|
||||
"result": "fail",
|
||||
"observations": [
|
||||
"CMIS Browser Binding endpoint is reachable, but declared capabilities are not supported by repository capability flags.",
|
||||
"Unsupported declared requirements: "
|
||||
+ ", ".join(item["requirement_ref"] for item in unsupported)
|
||||
+ ".",
|
||||
],
|
||||
"facts": facts,
|
||||
"artifact_refs": artifact_refs,
|
||||
}
|
||||
|
||||
observations = [
|
||||
"CMIS Browser Binding endpoint is reachable and returned parseable JSON."
|
||||
]
|
||||
if expected_gaps:
|
||||
observations.append(
|
||||
"Unsupported optional capabilities were accepted as known gaps: "
|
||||
+ ", ".join(item["requirement_ref"] for item in expected_gaps)
|
||||
+ "."
|
||||
)
|
||||
return {
|
||||
"result": "pass",
|
||||
"observations": [
|
||||
"CMIS Browser Binding endpoint is reachable and returned parseable JSON."
|
||||
],
|
||||
"observations": observations,
|
||||
"facts": facts,
|
||||
"artifact_refs": artifact_refs,
|
||||
}
|
||||
@@ -150,29 +181,40 @@ def _parse_json(body: bytes) -> Any:
|
||||
return None
|
||||
|
||||
|
||||
def _repository_facts(value: Any) -> dict[str, Any]:
|
||||
def _repository_facts(value: Any, context: dict[str, Any]) -> dict[str, Any]:
|
||||
if not isinstance(value, dict):
|
||||
return {"repository_shape": "unknown"}
|
||||
|
||||
if "repositoryId" in value:
|
||||
repository_id = str(value["repositoryId"])
|
||||
return {
|
||||
"repository_shape": "single-repository-info",
|
||||
"repository_ids": [value["repositoryId"]],
|
||||
"repository_ids": [repository_id],
|
||||
"selected_repository_id": repository_id,
|
||||
"cmis_version_supported": value.get("cmisVersionSupported"),
|
||||
"capabilities_present": isinstance(value.get("capabilities"), dict),
|
||||
"capability_flags": _capability_flags(value),
|
||||
"capability_posture": _capability_posture(value, context),
|
||||
}
|
||||
|
||||
repository_ids = []
|
||||
repositories: dict[str, dict[str, Any]] = {}
|
||||
for key, child in value.items():
|
||||
if isinstance(child, dict) and (
|
||||
"repositoryId" in child or "repositoryName" in child
|
||||
):
|
||||
repository_ids.append(str(child.get("repositoryId", key)))
|
||||
repositories[str(child.get("repositoryId", key))] = child
|
||||
|
||||
if repository_ids:
|
||||
if repositories:
|
||||
selected_repository_id = _selected_repository_id(repositories, context)
|
||||
selected_repository = repositories[selected_repository_id]
|
||||
return {
|
||||
"repository_shape": "repository-map",
|
||||
"repository_ids": repository_ids,
|
||||
"repository_ids": sorted(repositories),
|
||||
"selected_repository_id": selected_repository_id,
|
||||
"cmis_version_supported": selected_repository.get("cmisVersionSupported"),
|
||||
"capabilities_present": isinstance(selected_repository.get("capabilities"), dict),
|
||||
"capability_flags": _capability_flags(selected_repository),
|
||||
"capability_posture": _capability_posture(selected_repository, context),
|
||||
}
|
||||
|
||||
return {
|
||||
@@ -181,6 +223,99 @@ def _repository_facts(value: Any) -> dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def _selected_repository_id(
|
||||
repositories: dict[str, dict[str, Any]],
|
||||
context: dict[str, Any],
|
||||
) -> str:
|
||||
configured = _opencmis_policy(context).get("repository_id")
|
||||
if isinstance(configured, str) and configured in repositories:
|
||||
return configured
|
||||
return sorted(repositories)[0]
|
||||
|
||||
|
||||
def _capability_flags(repository_info: dict[str, Any]) -> dict[str, Any]:
|
||||
capabilities = repository_info.get("capabilities", {})
|
||||
return dict(capabilities) if isinstance(capabilities, dict) else {}
|
||||
|
||||
|
||||
def _capability_posture(
|
||||
repository_info: dict[str, Any],
|
||||
context: dict[str, Any],
|
||||
) -> list[dict[str, Any]]:
|
||||
target = context["target_profile"]
|
||||
declared = set(target.get("declared_capabilities", []))
|
||||
known_gap_refs = {
|
||||
requirement_ref: gap["id"]
|
||||
for gap in target.get("known_gaps", [])
|
||||
for requirement_ref in gap.get("requirement_refs", [])
|
||||
}
|
||||
refs = sorted(declared | set(known_gap_refs))
|
||||
flags = _capability_flags(repository_info)
|
||||
posture = []
|
||||
for requirement_ref in refs:
|
||||
support = _requirement_support(requirement_ref, flags)
|
||||
if support is True:
|
||||
status = "supported"
|
||||
elif support is False and requirement_ref in known_gap_refs:
|
||||
status = "expected_gap"
|
||||
elif support is False:
|
||||
status = "unsupported"
|
||||
else:
|
||||
status = "unknown"
|
||||
posture.append(
|
||||
{
|
||||
"requirement_ref": requirement_ref,
|
||||
"status": status,
|
||||
"known_gap_ref": known_gap_refs.get(requirement_ref),
|
||||
"flag_refs": _flag_refs(requirement_ref),
|
||||
}
|
||||
)
|
||||
return posture
|
||||
|
||||
|
||||
def _requirement_support(requirement_ref: str, flags: dict[str, Any]) -> bool | None:
|
||||
if requirement_ref == "cmis.repository-info":
|
||||
return True
|
||||
flag_refs = _flag_refs(requirement_ref)
|
||||
if not flag_refs:
|
||||
return None
|
||||
values = [flags[key] for key in flag_refs if key in flags]
|
||||
if not values:
|
||||
return None
|
||||
return any(_flag_supports(value) for value in values)
|
||||
|
||||
|
||||
def _flag_refs(requirement_ref: str) -> list[str]:
|
||||
return {
|
||||
"cmis.query": ["capabilityQuery"],
|
||||
"cmis.acl": ["capabilityACL"],
|
||||
"cmis.navigation-services": [
|
||||
"capabilityGetDescendants",
|
||||
"capabilityGetFolderTree",
|
||||
],
|
||||
"cmis.relationships": ["capabilityJoin"],
|
||||
"cmis.change-log": ["capabilityChanges"],
|
||||
"cmis.versioning": [
|
||||
"capabilityVersionSpecificFiling",
|
||||
"capabilityPWCSearchable",
|
||||
"capabilityPWCUpdatable",
|
||||
],
|
||||
}.get(requirement_ref, [])
|
||||
|
||||
|
||||
def _flag_supports(value: Any) -> bool:
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
return value.lower() not in {"", "false", "none", "no"}
|
||||
return value is not None
|
||||
|
||||
|
||||
def _opencmis_policy(context: dict[str, Any]) -> dict[str, Any]:
|
||||
policy = context["assessment_profile"].get("runtime_policy", {}).get("opencmis_tck", {})
|
||||
return policy if isinstance(policy, dict) else {}
|
||||
|
||||
|
||||
def _write_response_artifacts(
|
||||
context: dict[str, Any],
|
||||
status_code: int,
|
||||
|
||||
146
src/open_cmis_tck/profile.py
Normal file
146
src/open_cmis_tck/profile.py
Normal file
@@ -0,0 +1,146 @@
|
||||
"""CMIS-specific profile diagnostics for guide-board target profiles."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from urllib.parse import urlparse
|
||||
|
||||
|
||||
KNOWN_CMIS_REQUIREMENTS = {
|
||||
"cmis.repository-info",
|
||||
"cmis.type-definitions",
|
||||
"cmis.object-services",
|
||||
"cmis.content-streams",
|
||||
"cmis.navigation-services",
|
||||
"cmis.query",
|
||||
"cmis.relationships",
|
||||
"cmis.acl",
|
||||
"cmis.policies",
|
||||
"cmis.versioning",
|
||||
"cmis.change-log",
|
||||
"cmis.extensions",
|
||||
}
|
||||
|
||||
|
||||
def validate_cmis_profile_config(
|
||||
target_profile: dict[str, Any],
|
||||
assessment_profile: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Return actionable CMIS diagnostics for guide-board profiles.
|
||||
|
||||
The generic guide-board target profile remains the persisted contract. This
|
||||
helper explains how the OpenCMIS extension interprets those generic fields.
|
||||
"""
|
||||
|
||||
diagnostics: list[dict[str, str]] = []
|
||||
browser_endpoints = [
|
||||
endpoint
|
||||
for endpoint in target_profile.get("endpoints", [])
|
||||
if endpoint.get("binding") == "cmis-browser"
|
||||
]
|
||||
|
||||
if target_profile.get("subject_type") != "cmis-browser-binding-endpoint":
|
||||
diagnostics.append(
|
||||
_diagnostic(
|
||||
"warning",
|
||||
"subject_type",
|
||||
"CMIS targets should use subject_type 'cmis-browser-binding-endpoint'.",
|
||||
)
|
||||
)
|
||||
|
||||
if not browser_endpoints:
|
||||
diagnostics.append(
|
||||
_diagnostic(
|
||||
"error",
|
||||
"endpoints",
|
||||
"Add one endpoint with binding 'cmis-browser' and the Browser Binding service document URL.",
|
||||
)
|
||||
)
|
||||
for index, endpoint in enumerate(browser_endpoints):
|
||||
parsed = urlparse(str(endpoint.get("url", "")))
|
||||
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
|
||||
diagnostics.append(
|
||||
_diagnostic(
|
||||
"error",
|
||||
f"endpoints[{index}].url",
|
||||
"Use an absolute http(s) CMIS Browser Binding URL.",
|
||||
)
|
||||
)
|
||||
|
||||
declared = set(target_profile.get("declared_capabilities", []))
|
||||
unknown_declared = sorted(declared - KNOWN_CMIS_REQUIREMENTS)
|
||||
for requirement_ref in unknown_declared:
|
||||
diagnostics.append(
|
||||
_diagnostic(
|
||||
"warning",
|
||||
"declared_capabilities",
|
||||
f"Declared CMIS capability {requirement_ref!r} is not in the extension's known requirement list.",
|
||||
)
|
||||
)
|
||||
|
||||
known_gap_refs = {
|
||||
requirement_ref
|
||||
for gap in target_profile.get("known_gaps", [])
|
||||
for requirement_ref in gap.get("requirement_refs", [])
|
||||
}
|
||||
unexpected_gap_refs = sorted(known_gap_refs - KNOWN_CMIS_REQUIREMENTS)
|
||||
for requirement_ref in unexpected_gap_refs:
|
||||
diagnostics.append(
|
||||
_diagnostic(
|
||||
"warning",
|
||||
"known_gaps",
|
||||
f"Known gap {requirement_ref!r} is not in the extension's known requirement list.",
|
||||
)
|
||||
)
|
||||
|
||||
runtime_policy = (assessment_profile or {}).get("runtime_policy", {})
|
||||
opencmis_policy = runtime_policy.get("opencmis_tck", {})
|
||||
if opencmis_policy and not isinstance(opencmis_policy, dict):
|
||||
diagnostics.append(
|
||||
_diagnostic(
|
||||
"error",
|
||||
"runtime_policy.opencmis_tck",
|
||||
"OpenCMIS runtime policy must be an object when present.",
|
||||
)
|
||||
)
|
||||
opencmis_policy = {}
|
||||
|
||||
timeout = runtime_policy.get("timeout_seconds")
|
||||
if assessment_profile is not None and not isinstance(timeout, (int, float)):
|
||||
diagnostics.append(
|
||||
_diagnostic(
|
||||
"warning",
|
||||
"runtime_policy.timeout_seconds",
|
||||
"Set timeout_seconds to bound preflight and TCK execution.",
|
||||
)
|
||||
)
|
||||
|
||||
repository_id = None
|
||||
if isinstance(opencmis_policy, dict):
|
||||
repository_id = opencmis_policy.get("repository_id")
|
||||
if repository_id is not None and not isinstance(repository_id, str):
|
||||
diagnostics.append(
|
||||
_diagnostic(
|
||||
"error",
|
||||
"runtime_policy.opencmis_tck.repository_id",
|
||||
"repository_id must be a string when configured.",
|
||||
)
|
||||
)
|
||||
|
||||
status = "invalid" if any(item["severity"] == "error" for item in diagnostics) else "valid"
|
||||
return {
|
||||
"status": status,
|
||||
"diagnostics": diagnostics,
|
||||
"cmis_config": {
|
||||
"browser_binding_url": browser_endpoints[0]["url"] if browser_endpoints else None,
|
||||
"repository_id": repository_id,
|
||||
"auth_mode": "anonymous" if target_profile.get("credentials_ref") is None else "credentials_ref",
|
||||
"declared_capabilities": sorted(declared),
|
||||
"known_gap_refs": sorted(known_gap_refs),
|
||||
"timeout_seconds": timeout,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _diagnostic(severity: str, field: str, message: str) -> dict[str, str]:
|
||||
return {"severity": severity, "field": field, "message": message}
|
||||
@@ -1,7 +1,10 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import http.client
|
||||
import json
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from pathlib import Path
|
||||
@@ -10,6 +13,9 @@ from tempfile import TemporaryDirectory
|
||||
from guide_board.discovery import discover_extensions
|
||||
from guide_board.execution import run_assessment
|
||||
from guide_board.planning import build_run_plan, validate_assessment_profile
|
||||
from guide_board.retention import build_trend_summary
|
||||
from guide_board.service import ServiceHandle, start_service
|
||||
from open_cmis_tck.profile import validate_cmis_profile_config
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
@@ -44,6 +50,37 @@ class OpenCmisTckExtensionTests(unittest.TestCase):
|
||||
self.assertEqual(plan["extension_snapshots"][0]["path"], str(ROOT))
|
||||
self.assertEqual(len(plan["ordered_steps"]), 3)
|
||||
|
||||
def test_validates_cmis_profile_config_with_actionable_diagnostics(self) -> None:
|
||||
target = json.loads(
|
||||
(ROOT / "profiles" / "targets" / "kontextual-cmis-compat.json").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
)
|
||||
assessment = json.loads(
|
||||
(ROOT / "profiles" / "assessments" / "cmis-browser-baseline.json").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
)
|
||||
|
||||
diagnostics = validate_cmis_profile_config(target, assessment)
|
||||
|
||||
self.assertEqual(diagnostics["status"], "valid")
|
||||
self.assertEqual(
|
||||
diagnostics["cmis_config"]["browser_binding_url"],
|
||||
"http://127.0.0.1:8000/cmis/compat-tck/browser",
|
||||
)
|
||||
self.assertEqual(diagnostics["cmis_config"]["repository_id"], "compat-tck")
|
||||
self.assertEqual(diagnostics["cmis_config"]["auth_mode"], "anonymous")
|
||||
|
||||
broken = dict(target)
|
||||
broken["endpoints"] = []
|
||||
broken_diagnostics = validate_cmis_profile_config(broken, assessment)
|
||||
self.assertEqual(broken_diagnostics["status"], "invalid")
|
||||
self.assertIn(
|
||||
"Add one endpoint with binding 'cmis-browser'",
|
||||
broken_diagnostics["diagnostics"][0]["message"],
|
||||
)
|
||||
|
||||
def test_runs_cmis_preflight_against_local_endpoint(self) -> None:
|
||||
server = HTTPServer(("127.0.0.1", 0), _CmisHandler)
|
||||
thread = threading.Thread(target=server.serve_forever)
|
||||
@@ -88,6 +125,11 @@ class OpenCmisTckExtensionTests(unittest.TestCase):
|
||||
evidence[0]["facts"]["repository_ids"],
|
||||
["local-test-repository"],
|
||||
)
|
||||
posture = {
|
||||
item["requirement_ref"]: item["status"]
|
||||
for item in evidence[0]["facts"]["capability_posture"]
|
||||
}
|
||||
self.assertEqual(posture["cmis.repository-info"], "supported")
|
||||
self.assertEqual(len(package["artifact_manifest"]), 2)
|
||||
self.assertTrue(
|
||||
(
|
||||
@@ -103,6 +145,60 @@ class OpenCmisTckExtensionTests(unittest.TestCase):
|
||||
thread.join(timeout=5)
|
||||
server.server_close()
|
||||
|
||||
def test_preflight_accepts_unsupported_optional_capability_as_known_gap(self) -> None:
|
||||
server = HTTPServer(("127.0.0.1", 0), _CmisHandler)
|
||||
thread = threading.Thread(target=server.serve_forever)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
try:
|
||||
with TemporaryDirectory() as temporary_directory:
|
||||
temp_root = Path(temporary_directory)
|
||||
target_path = temp_root / "target.json"
|
||||
assessment_path = temp_root / "assessment.json"
|
||||
_write_target(target_path, server.server_port, "local-cmis-query-gap")
|
||||
target = json.loads(target_path.read_text(encoding="utf-8"))
|
||||
target["declared_capabilities"].append("cmis.query")
|
||||
target["known_gaps"].append(
|
||||
{
|
||||
"id": "query-not-targeted",
|
||||
"requirement_refs": ["cmis.query"],
|
||||
"reason": "The local fixture deliberately reports no query support.",
|
||||
"status": "unsupported_by_design",
|
||||
}
|
||||
)
|
||||
target_path.write_text(json.dumps(target), encoding="utf-8")
|
||||
_write_assessment(
|
||||
assessment_path,
|
||||
"local-cmis-known-gap",
|
||||
"local-cmis-query-gap",
|
||||
[],
|
||||
None,
|
||||
)
|
||||
|
||||
result = run_assessment(
|
||||
CORE_ROOT,
|
||||
target_path,
|
||||
assessment_path,
|
||||
temp_root / "run",
|
||||
[ROOT],
|
||||
)
|
||||
evidence = json.loads(
|
||||
(Path(result["run_dir"]) / "normalized" / "evidence.json").read_text(
|
||||
encoding="utf-8"
|
||||
)
|
||||
)["evidence"]
|
||||
posture = {
|
||||
item["requirement_ref"]: item["status"]
|
||||
for item in evidence[0]["facts"]["capability_posture"]
|
||||
}
|
||||
|
||||
self.assertEqual(result["status"], "completed")
|
||||
self.assertEqual(posture["cmis.query"], "expected_gap")
|
||||
finally:
|
||||
server.shutdown()
|
||||
thread.join(timeout=5)
|
||||
server.server_close()
|
||||
|
||||
def test_runs_cmis_tck_command_wrapper_boundary(self) -> None:
|
||||
server = HTTPServer(("127.0.0.1", 0), _CmisHandler)
|
||||
thread = threading.Thread(target=server.serve_forever)
|
||||
@@ -163,6 +259,144 @@ class OpenCmisTckExtensionTests(unittest.TestCase):
|
||||
thread.join(timeout=5)
|
||||
server.server_close()
|
||||
|
||||
def test_runs_configured_tck_command_and_normalizes_json_results(self) -> None:
|
||||
server = HTTPServer(("127.0.0.1", 0), _CmisHandler)
|
||||
thread = threading.Thread(target=server.serve_forever)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
try:
|
||||
with TemporaryDirectory() as temporary_directory:
|
||||
temp_root = Path(temporary_directory)
|
||||
target_path = temp_root / "target.json"
|
||||
assessment_path = temp_root / "assessment.json"
|
||||
fake_tck = temp_root / "fake_tck.py"
|
||||
fake_tck.write_text(
|
||||
"\n".join(
|
||||
[
|
||||
"import json",
|
||||
"print(json.dumps({",
|
||||
" 'tests': [",
|
||||
" {'id': 'repository-info', 'status': 'pass'},",
|
||||
" {'id': 'type-definitions', 'status': 'pass'}",
|
||||
" ]",
|
||||
"}))",
|
||||
]
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
_write_target(target_path, server.server_port, "local-cmis-configured-tck")
|
||||
_write_assessment(
|
||||
assessment_path,
|
||||
"local-cmis-configured-tck",
|
||||
"local-cmis-configured-tck",
|
||||
["repository-type"],
|
||||
None,
|
||||
{
|
||||
"requires_java_maven": False,
|
||||
"repository_id": "local-test-repository",
|
||||
"command": [
|
||||
sys.executable,
|
||||
str(fake_tck),
|
||||
"--url",
|
||||
"{browser_url}",
|
||||
"--repository",
|
||||
"{repository_id}",
|
||||
"--group",
|
||||
"{check_group}",
|
||||
],
|
||||
},
|
||||
)
|
||||
|
||||
result = run_assessment(
|
||||
CORE_ROOT,
|
||||
target_path,
|
||||
assessment_path,
|
||||
temp_root / "run",
|
||||
[ROOT],
|
||||
)
|
||||
run_dir = Path(result["run_dir"])
|
||||
evidence = json.loads(
|
||||
(run_dir / "normalized" / "evidence.json").read_text(encoding="utf-8")
|
||||
)["evidence"]
|
||||
retention = json.loads(
|
||||
(run_dir / "retention-summary.json").read_text(encoding="utf-8")
|
||||
)
|
||||
trend = build_trend_summary(temp_root)
|
||||
|
||||
self.assertEqual(result["status"], "completed")
|
||||
self.assertEqual(evidence[1]["result"], "pass")
|
||||
self.assertEqual(evidence[1]["facts"]["normalizer"], "json-cases")
|
||||
self.assertEqual(evidence[1]["facts"]["result_counts"], {"pass": 2})
|
||||
self.assertTrue(
|
||||
(
|
||||
run_dir
|
||||
/ "artifacts"
|
||||
/ "open-cmis-tck"
|
||||
/ "tck"
|
||||
/ "repository-type"
|
||||
/ "stdout.log"
|
||||
).exists()
|
||||
)
|
||||
self.assertEqual(retention["summary"]["status"], "completed")
|
||||
self.assertGreaterEqual(retention["summary"]["artifact_count"], 4)
|
||||
self.assertEqual(trend["run_count"], 1)
|
||||
finally:
|
||||
server.shutdown()
|
||||
thread.join(timeout=5)
|
||||
server.server_close()
|
||||
|
||||
def test_guide_board_service_runs_cmis_extension(self) -> None:
|
||||
server = HTTPServer(("127.0.0.1", 0), _CmisHandler)
|
||||
thread = threading.Thread(target=server.serve_forever)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
service = start_service(CORE_ROOT, [ROOT], host="127.0.0.1", port=0)
|
||||
try:
|
||||
with TemporaryDirectory() as temporary_directory:
|
||||
temp_root = Path(temporary_directory)
|
||||
target_path = temp_root / "target.json"
|
||||
assessment_path = temp_root / "assessment.json"
|
||||
_write_target(target_path, server.server_port, "local-cmis-service")
|
||||
_write_assessment(
|
||||
assessment_path,
|
||||
"local-cmis-service",
|
||||
"local-cmis-service",
|
||||
[],
|
||||
None,
|
||||
)
|
||||
|
||||
extensions = _request_json(service, "GET", "/extensions")
|
||||
self.assertIn(
|
||||
"open-cmis-tck",
|
||||
[extension["id"] for extension in extensions["extensions"]],
|
||||
)
|
||||
|
||||
job = _request_json(
|
||||
service,
|
||||
"POST",
|
||||
"/runs",
|
||||
{
|
||||
"target": str(target_path),
|
||||
"assessment": str(assessment_path),
|
||||
"output_dir": str(temp_root / "service-run"),
|
||||
},
|
||||
expected_status=202,
|
||||
)
|
||||
status = _wait_for_job(service, job["job_id"])
|
||||
reports = _request_json(service, "GET", f"/runs/{job['job_id']}/reports")
|
||||
|
||||
self.assertEqual(status["status"], "succeeded")
|
||||
self.assertEqual(status["result"]["status"], "completed")
|
||||
self.assertEqual(
|
||||
reports["assessment_package"]["json"]["extensions"][0]["id"],
|
||||
"open-cmis-tck",
|
||||
)
|
||||
finally:
|
||||
service.stop()
|
||||
server.shutdown()
|
||||
thread.join(timeout=5)
|
||||
server.server_close()
|
||||
|
||||
def test_preflight_failure_blocks_downstream_checks(self) -> None:
|
||||
with TemporaryDirectory() as temporary_directory:
|
||||
temp_root = Path(temporary_directory)
|
||||
@@ -259,7 +493,14 @@ def _write_assessment(
|
||||
target_id: str,
|
||||
check_groups: list[str],
|
||||
waiver_ref: str | None,
|
||||
opencmis_policy: dict[str, object] | None = None,
|
||||
) -> None:
|
||||
runtime_policy: dict[str, object] = {
|
||||
"offline": False,
|
||||
"timeout_seconds": 15,
|
||||
}
|
||||
if opencmis_policy is not None:
|
||||
runtime_policy["opencmis_tck"] = opencmis_policy
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
@@ -278,10 +519,7 @@ def _write_assessment(
|
||||
"summary_days": 365,
|
||||
"raw_artifact_days": 0,
|
||||
},
|
||||
"runtime_policy": {
|
||||
"offline": False,
|
||||
"timeout_seconds": 15,
|
||||
},
|
||||
"runtime_policy": runtime_policy,
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
@@ -316,6 +554,42 @@ def _write_command_waiver(path: Path, target_id: str) -> None:
|
||||
)
|
||||
|
||||
|
||||
def _request_json(
|
||||
service: ServiceHandle,
|
||||
method: str,
|
||||
path: str,
|
||||
payload: dict[str, object] | None = None,
|
||||
expected_status: int = 200,
|
||||
) -> dict[str, object]:
|
||||
connection = http.client.HTTPConnection(service.host, service.port, timeout=5)
|
||||
body = None
|
||||
headers = {}
|
||||
if payload is not None:
|
||||
body = json.dumps(payload).encode("utf-8")
|
||||
headers["Content-Type"] = "application/json"
|
||||
try:
|
||||
connection.request(method, path, body=body, headers=headers)
|
||||
response = connection.getresponse()
|
||||
data = response.read().decode("utf-8")
|
||||
finally:
|
||||
connection.close()
|
||||
if response.status != expected_status:
|
||||
raise AssertionError(f"expected HTTP {expected_status}, got {response.status}: {data}")
|
||||
value = json.loads(data)
|
||||
if not isinstance(value, dict):
|
||||
raise AssertionError(f"expected JSON object response, got {type(value).__name__}")
|
||||
return value
|
||||
|
||||
|
||||
def _wait_for_job(service: ServiceHandle, job_id: str) -> dict[str, object]:
|
||||
for _ in range(50):
|
||||
status = _request_json(service, "GET", f"/runs/{job_id}")
|
||||
if status["status"] in {"succeeded", "failed"}:
|
||||
return status
|
||||
time.sleep(0.05)
|
||||
raise AssertionError(f"job did not finish: {job_id}")
|
||||
|
||||
|
||||
class _CmisHandler(BaseHTTPRequestHandler):
|
||||
def do_GET(self) -> None:
|
||||
body = json.dumps(
|
||||
@@ -324,7 +598,13 @@ class _CmisHandler(BaseHTTPRequestHandler):
|
||||
"repositoryId": "local-test-repository",
|
||||
"repositoryName": "Local Test Repository",
|
||||
"cmisVersionSupported": "1.1",
|
||||
"capabilities": {},
|
||||
"capabilities": {
|
||||
"capabilityACL": "discover",
|
||||
"capabilityChanges": "none",
|
||||
"capabilityGetDescendants": True,
|
||||
"capabilityGetFolderTree": True,
|
||||
"capabilityQuery": "none",
|
||||
},
|
||||
}
|
||||
}
|
||||
).encode("utf-8")
|
||||
|
||||
@@ -2,10 +2,10 @@
|
||||
id: OPEN-CMIS-TCK-WP-0001
|
||||
type: extension-workplan
|
||||
title: "OpenCMIS TCK Harness Foundation"
|
||||
repo: guide-board
|
||||
repo: open-cmis-tck
|
||||
extension: open-cmis-tck
|
||||
domain: markitect
|
||||
status: active
|
||||
status: completed
|
||||
owner: codex
|
||||
planning_priority: high
|
||||
planning_order: 2
|
||||
@@ -81,7 +81,7 @@ Progress:
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T002
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "2ccc74a7-bed9-4769-8608-d579fdf3a0cd"
|
||||
```
|
||||
@@ -95,11 +95,19 @@ Acceptance:
|
||||
- Profile validation produces actionable diagnostics for missing or invalid
|
||||
fields.
|
||||
|
||||
Progress:
|
||||
|
||||
- Added CMIS-specific profile diagnostics in `open_cmis_tck.profile`.
|
||||
- Documented the CMIS profile contract in `docs/CMIS-PROFILES.md`.
|
||||
- Added `runtime_policy.opencmis_tck.repository_id` to the baseline assessment
|
||||
profile while keeping the persisted profiles compatible with guide-board core
|
||||
schemas.
|
||||
|
||||
## D1.3 - CMIS Preflight Probe
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T003
|
||||
status: in_progress
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "6d45885b-78a4-4e8b-8fcc-b8d6488e703b"
|
||||
```
|
||||
@@ -119,14 +127,15 @@ Progress:
|
||||
assessment-package fingerprinting.
|
||||
- Failed CMIS preflight now blocks downstream OpenCMIS TCK groups instead of
|
||||
invoking the Java/Maven wrapper against an invalid target.
|
||||
- Capability flag normalization remains to be expanded after a live target sample
|
||||
is captured.
|
||||
- Capability flags are normalized into repository capability posture facts.
|
||||
- Unsupported optional capabilities can be accepted as target-profile known gaps
|
||||
without hiding unexpected unsupported declared capabilities.
|
||||
|
||||
## D1.4 - OpenCMIS TCK Runner Wrapper
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T004
|
||||
status: in_progress
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "502d7586-6f9e-475e-9683-43260666d5d9"
|
||||
```
|
||||
@@ -145,14 +154,17 @@ Progress:
|
||||
evidence when dependencies or final TCK invocation details are missing.
|
||||
- `profiles/expectations/cmis-local-harness.json` marks local bootstrap blockers
|
||||
as expected without hiding target preflight failures.
|
||||
- Actual Apache Chemistry TCK classpath resolution, group invocation, and raw log
|
||||
capture remain to be implemented.
|
||||
- The wrapper can invoke a configured argv command from
|
||||
`runtime_policy.opencmis_tck.command` or environment variables.
|
||||
- The wrapper expands CMIS placeholders, captures stdout/stderr and invocation
|
||||
metadata under the run directory, and skips cleanly when Java/Maven or command
|
||||
configuration are missing.
|
||||
|
||||
## D1.5 - CMIS Result Normalization
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T005
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "716486b6-6f14-41f8-8417-5015ba746005"
|
||||
```
|
||||
@@ -165,11 +177,19 @@ Acceptance:
|
||||
- Failures include enough context to map back to TCK group, capability group,
|
||||
target profile, and raw artifact paths.
|
||||
|
||||
Progress:
|
||||
|
||||
- Added runner-side normalization for JSON case lists, JSON runner results,
|
||||
JUnit-style XML, and exit-code-only output.
|
||||
- Normalized result counts, case IDs, selected check group, target URL,
|
||||
repository ID, return code, and artifact paths are captured in guide-board
|
||||
evidence facts.
|
||||
|
||||
## D1.6 - Capability Mapping And Reports
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T006
|
||||
status: in_progress
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "9f7dacc5-4d19-4755-aa9a-8572d4285514"
|
||||
```
|
||||
@@ -190,12 +210,16 @@ Progress:
|
||||
groups.
|
||||
- Guide-board writes normalized mapping records and includes capability-group
|
||||
counts in Markdown reports.
|
||||
- Mapping coverage now includes relationships, change log, policy, and extension
|
||||
gap requirement refs.
|
||||
- Additional check groups are declared for relationships, change log, and
|
||||
extension/known-gap review.
|
||||
|
||||
## D1.7 - Optional Local Service API Adapter
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T007
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "a05e47bd-88db-4878-aef4-bf328790c3f0"
|
||||
```
|
||||
@@ -207,11 +231,17 @@ Acceptance:
|
||||
- CLI operation remains the primary path.
|
||||
- Long-running TCK jobs are tracked without blocking the API process.
|
||||
|
||||
Progress:
|
||||
|
||||
- Verified `open-cmis-tck` through guide-board's local service as an external
|
||||
extension.
|
||||
- Documented service usage in `docs/SERVICE-AND-RETENTION.md`.
|
||||
|
||||
## D1.8 - Historical Result Retention
|
||||
|
||||
```task
|
||||
id: OPEN-CMIS-TCK-WP-0001-T008
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "c27ea43f-41ec-49d0-a890-3681455f7c6c"
|
||||
```
|
||||
@@ -223,6 +253,14 @@ Acceptance:
|
||||
- Summaries are suitable for trend charts and downstream capability-score
|
||||
updates.
|
||||
|
||||
Progress:
|
||||
|
||||
- CMIS runs now exercise guide-board retention summaries and trend input through
|
||||
extension tests.
|
||||
- The sample assessment profile declares 365-day summary retention and 30-day
|
||||
raw artifact retention.
|
||||
- Retention behavior is documented in `docs/SERVICE-AND-RETENTION.md`.
|
||||
|
||||
## Definition Of Done
|
||||
|
||||
- A developer can configure a CMIS Browser Binding endpoint.
|
||||
|
||||
Reference in New Issue
Block a user