diff --git a/docs/ASSESSMENT-OPERATIONS.md b/docs/ASSESSMENT-OPERATIONS.md index 4a7ef6b..ac8083c 100644 --- a/docs/ASSESSMENT-OPERATIONS.md +++ b/docs/ASSESSMENT-OPERATIONS.md @@ -164,6 +164,15 @@ Fetch reports after the job status is `succeeded`: curl -sf http://127.0.0.1:8080/runs/JOB_ID/reports | python3 -m json.tool ``` +Inspect retained run history, including runs produced before the current +service process started: + +```sh +curl -sf "http://127.0.0.1:8080/retained-runs?runs_dir=runs" | python3 -m json.tool +curl -sf "http://127.0.0.1:8080/retained-runs/latest?runs_dir=runs" | python3 -m json.tool +curl -sf "http://127.0.0.1:8080/retained-runs/RUN_ID/artifact-manifest?runs_dir=runs" | python3 -m json.tool +``` + Service job state is currently in memory for the running service process. Run artifacts are durable in the output directory and can still be inspected after a service restart. See `docs/SERVICE-JOB-DURABILITY.md` for the restart and diff --git a/docs/CONTAINER.md b/docs/CONTAINER.md index 3d5ecc7..c698424 100644 --- a/docs/CONTAINER.md +++ b/docs/CONTAINER.md @@ -144,4 +144,9 @@ podman run --rm -p 8080:8080 \ The service layer adds in-memory job tracking and HTTP transport. Execution semantics remain the CLI/core semantics documented in -`docs/LOCAL-SERVICE-API.md`. +`docs/LOCAL-SERVICE-API.md`. Mounted run directories remain discoverable through +the retained-run endpoints, for example: + +```sh +curl -sf "http://127.0.0.1:8080/retained-runs?runs_dir=/runs" | python3 -m json.tool +``` diff --git a/docs/LOCAL-SERVICE-API.md b/docs/LOCAL-SERVICE-API.md index 481142f..2f687ec 100644 --- a/docs/LOCAL-SERVICE-API.md +++ b/docs/LOCAL-SERVICE-API.md @@ -98,7 +98,41 @@ errors. ### `GET /runs/{job_id}/reports` Returns the Markdown report content, assessment package JSON, retention summary, -and their filesystem paths after a job has succeeded. +submission package JSON when present, and their filesystem paths after a job has +succeeded. + +### `GET /retained-runs` + +Lists durable retained run summaries by scanning a runs directory. Without a +query parameter, the service scans `/runs`. + +```text +GET /retained-runs?runs_dir=/runs +``` + +### `GET /retained-runs/latest` + +Selects the latest retained run, optionally filtered by target and assessment +profile refs. + +```text +GET /retained-runs/latest?runs_dir=/runs&target=sample-repository&assessment=sample-noop-assessment +``` + +### `GET /retained-runs/{run_id}/reports` + +Returns the retained summary plus safe report paths for a durable run. This +works after a service restart because it reads `retention-summary.json` from +disk instead of in-memory job records. + +### `GET /retained-runs/{run_id}/artifact-manifest` + +Returns the assessment package `artifact_manifest` for a retained run. If the +run predates assessment packages, the response is compatible and returns an +empty manifest with `compatibility: "assessment-package-missing"`. + +Retained-run endpoints validate report and artifact paths before returning +them. A path that escapes the selected run directory is rejected. ## Container Mode @@ -112,5 +146,6 @@ podman run --rm -p 8080:8080 \ ``` The service keeps job state in memory. Durable run evidence remains in the -mounted output directory. See `docs/SERVICE-JOB-DURABILITY.md` for the explicit -restart and recovery contract. +mounted output directory and can be discovered through `GET /retained-runs` +after restart. See `docs/SERVICE-JOB-DURABILITY.md` for the explicit recovery +contract. diff --git a/docs/SERVICE-JOB-DURABILITY.md b/docs/SERVICE-JOB-DURABILITY.md index e206f0f..c3883eb 100644 --- a/docs/SERVICE-JOB-DURABILITY.md +++ b/docs/SERVICE-JOB-DURABILITY.md @@ -13,16 +13,19 @@ Durable state lives in run directories: - `run.json` - `plan.json` +- `sources.lock.json` - `retention-summary.json` - `normalized/evidence.json` - `normalized/findings.json` - `normalized/mappings.json` - `reports/assessment-package.json` - `reports/report.md` +- `reports/submission-package.json` - `artifacts/` The durable recovery index is the set of `retention-summary.json` files under a -runs directory. +runs directory. No separate durable service index is required for the baseline; +the service reconstructs retained-run views by scanning those summaries. ## Why In-Memory Jobs Stay The Baseline @@ -47,9 +50,14 @@ After a service restart: - old `job_id` values are invalid, - `GET /runs/{job_id}` cannot recover pre-restart job metadata, - `GET /runs/{job_id}/reports` only works for jobs known to the current process, -- run artifacts from earlier service processes remain available on disk. +- run artifacts from earlier service processes remain available on disk, +- `GET /retained-runs`, `GET /retained-runs/latest`, + `GET /retained-runs/{run_id}/reports`, and + `GET /retained-runs/{run_id}/artifact-manifest` can expose completed retained + runs after restart. -Operators should recover previous results with the CLI run-history commands: +Operators can recover previous results with either the CLI run-history commands +or the retained-run service endpoints: ```sh PYTHONPATH=src python3 -m guide_board runs list --runs-dir runs @@ -57,6 +65,12 @@ PYTHONPATH=src python3 -m guide_board runs latest --runs-dir runs PYTHONPATH=src python3 -m guide_board runs report --runs-dir runs --run-id RUN_ID ``` +```sh +curl -sf "http://127.0.0.1:8080/retained-runs?runs_dir=runs" | python3 -m json.tool +curl -sf "http://127.0.0.1:8080/retained-runs/RUN_ID/reports?runs_dir=runs" | python3 -m json.tool +curl -sf "http://127.0.0.1:8080/retained-runs/RUN_ID/artifact-manifest?runs_dir=runs" | python3 -m json.tool +``` + ## Recovery Flow Use this flow when the service process restarted or a browser/UI lost its job @@ -64,8 +78,9 @@ state: 1. Identify the output directory passed to `POST /runs`. 2. Confirm whether `retention-summary.json` exists. -3. If it exists, use `guide-board runs report --runs-dir ` to retrieve - report paths. +3. If it exists, use `guide-board runs report --runs-dir ` or + `GET /retained-runs/{run_id}/reports?runs_dir=` to retrieve report + paths. 4. If only partial files exist, inspect `run.json`, `plan.json`, and artifacts before rerunning. 5. Rerun into a fresh output directory when the prior status is unclear. @@ -73,8 +88,8 @@ state: ## Future Durable Index Option A future durable service index may be added if UI or automation workflows need -cross-restart job lookup. If added, it should remain reconstructable from run -directories and should not become the authority for assessment results. +cross-restart transport job lookup. If added, it should remain reconstructable +from run directories and should not become the authority for assessment results. The minimum acceptable durable index would contain: diff --git a/scripts/container_smoke.sh b/scripts/container_smoke.sh index f57526d..2fd9692 100755 --- a/scripts/container_smoke.sh +++ b/scripts/container_smoke.sh @@ -35,12 +35,14 @@ echo "==> Verifying mounted run artifacts" for path in \ "$RUNS_DIR/sample-noop/run.json" \ "$RUNS_DIR/sample-noop/plan.json" \ + "$RUNS_DIR/sample-noop/sources.lock.json" \ "$RUNS_DIR/sample-noop/retention-summary.json" \ "$RUNS_DIR/sample-noop/normalized/evidence.json" \ "$RUNS_DIR/sample-noop/normalized/findings.json" \ "$RUNS_DIR/sample-noop/normalized/mappings.json" \ "$RUNS_DIR/sample-noop/reports/assessment-package.json" \ - "$RUNS_DIR/sample-noop/reports/report.md" + "$RUNS_DIR/sample-noop/reports/report.md" \ + "$RUNS_DIR/sample-noop/reports/submission-package.json" do if [ ! -f "$path" ]; then echo "ERROR: expected artifact missing: $path" >&2 diff --git a/src/guide_board/service.py b/src/guide_board/service.py index 5ab4a25..a993cf0 100644 --- a/src/guide_board/service.py +++ b/src/guide_board/service.py @@ -10,7 +10,7 @@ from datetime import datetime, timezone from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from pathlib import Path from typing import Any -from urllib.parse import urlparse +from urllib.parse import parse_qs, unquote, urlparse from guide_board.discovery import discover_extensions from guide_board.errors import GuideBoardError @@ -21,6 +21,11 @@ from guide_board.planning import ( validate_assessment_profile, validate_target_profile, ) +from guide_board.retention import ( + list_retained_runs, + retained_run_report_paths, + select_retained_run, +) @dataclass(frozen=True) @@ -131,7 +136,7 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): def _handle(self, method: str) -> None: parsed = urlparse(self.path) try: - response, status_code = self._route(method, parsed.path) + response, status_code = self._route(method, parsed.path, parsed.query) except HttpProblem as exc: response = _error_response(exc.message, exc.__class__.__name__, exc.status_code) status_code = exc.status_code @@ -147,7 +152,8 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): self._send_json(status_code, response) - def _route(self, method: str, path: str) -> tuple[dict[str, Any], int]: + def _route(self, method: str, path: str, query: str = "") -> tuple[dict[str, Any], int]: + query_params = _query_params(query) if method == "GET" and path == "/health": return self._health(), 200 if method == "GET" and path == "/extensions": @@ -160,6 +166,10 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): return {"runs": self.server.context.jobs.list()}, 200 if method == "POST" and path == "/runs": return self._start_run(), 202 + if method == "GET" and path == "/retained-runs": + return self._retained_runs(query_params), 200 + if method == "GET" and path == "/retained-runs/latest": + return self._retained_latest(query_params), 200 run_match = _match_run_path(path) if method == "GET" and run_match is not None: @@ -169,6 +179,14 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): if suffix == "reports": return self._run_reports(job_id), 200 + retained_match = _match_retained_run_path(path) + if method == "GET" and retained_match is not None: + run_id, suffix = retained_match + if suffix == "reports": + return self._retained_run_reports(run_id, query_params), 200 + if suffix == "artifact-manifest": + return self._retained_artifact_manifest(run_id, query_params), 200 + raise HttpProblem(404, f"endpoint not found: {method} {path}") def _health(self) -> dict[str, Any]: @@ -261,10 +279,21 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): report_path = Path(result["report"]) package_path = Path(result["assessment_package"]) retention_path = Path(result["retention_summary"]) + submission_value = result.get("submission_package") + submission_path = ( + Path(submission_value) + if isinstance(submission_value, str) and submission_value + else None + ) try: report_markdown = report_path.read_text(encoding="utf-8") assessment_package = load_json(package_path) retention_summary = load_json(retention_path) + submission_package = ( + load_json(submission_path) + if submission_path is not None and submission_path.is_file() + else None + ) except OSError as exc: raise HttpProblem(404, f"run report artifact is missing: {exc}") from exc @@ -277,6 +306,7 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): "report": str(report_path), "assessment_package": str(package_path), "retention_summary": str(retention_path), + "submission_package": str(submission_path) if submission_path is not None else None, }, "report": { "path": str(report_path), @@ -290,6 +320,69 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): "path": str(retention_path), "json": retention_summary, }, + "submission_package": { + "path": str(submission_path) if submission_package else None, + "json": submission_package, + }, + } + + def _retained_runs(self, query: dict[str, str]) -> dict[str, Any]: + runs_dir = _runs_dir_from_query(self.server.context.root, query) + return { + "runs_dir": str(runs_dir), + "runs": list_retained_runs(runs_dir), + } + + def _retained_latest(self, query: dict[str, str]) -> dict[str, Any]: + runs_dir = _runs_dir_from_query(self.server.context.root, query) + run = select_retained_run( + runs_dir, + target_profile_ref=query.get("target"), + assessment_profile_ref=query.get("assessment"), + ) + return { + "runs_dir": str(runs_dir), + "selection": { + "target_profile_ref": query.get("target"), + "assessment_profile_ref": query.get("assessment"), + }, + "run": _retained_run_with_paths(run) if run else None, + } + + def _retained_run_reports(self, run_id: str, query: dict[str, str]) -> dict[str, Any]: + runs_dir = _runs_dir_from_query(self.server.context.root, query) + run = _select_retained_run_or_404(runs_dir, run_id) + return { + "runs_dir": str(runs_dir), + "run": _retained_run_with_paths(run), + } + + def _retained_artifact_manifest(self, run_id: str, query: dict[str, str]) -> dict[str, Any]: + runs_dir = _runs_dir_from_query(self.server.context.root, query) + run = _select_retained_run_or_404(runs_dir, run_id) + run_dir = _safe_run_dir(runs_dir, run) + package_path = run_dir / "reports" / "assessment-package.json" + if not package_path.exists(): + return { + "runs_dir": str(runs_dir), + "run_id": run_id, + "run_dir": str(run_dir), + "artifact_manifest": [], + "compatibility": "assessment-package-missing", + } + package = load_json(package_path) + artifacts = package.get("artifact_manifest", []) + if not isinstance(artifacts, list): + raise HttpProblem(400, f"{package_path}: artifact_manifest must be a list") + for artifact in artifacts: + if isinstance(artifact, dict): + _safe_run_ref(run_dir, artifact.get("path")) + return { + "runs_dir": str(runs_dir), + "run_id": run_id, + "run_dir": str(run_dir), + "artifact_manifest": artifacts, + "compatibility": "current", } def _read_payload(self) -> dict[str, Any]: @@ -430,6 +523,81 @@ def _match_run_path(path: str) -> tuple[str, str | None] | None: return None +def _match_retained_run_path(path: str) -> tuple[str, str] | None: + parts = [unquote(part) for part in path.split("/") if part] + if len(parts) == 3 and parts[0] == "retained-runs": + return parts[1], parts[2] + return None + + +def _query_params(query: str) -> dict[str, str]: + parsed = parse_qs(query, keep_blank_values=False) + params = {} + for key, values in parsed.items(): + if values: + params[key] = values[-1] + return params + + +def _runs_dir_from_query(root: Path, query: dict[str, str]) -> Path: + runs_dir = query.get("runs_dir") + if not runs_dir: + return (root / "runs").resolve() + return _resolve_path(root, runs_dir) + + +def _select_retained_run_or_404(runs_dir: Path, run_id: str) -> dict[str, Any]: + run = select_retained_run(runs_dir, run_id=run_id) + if run is None: + raise HttpProblem(404, f"retained run not found: {run_id}") + return run + + +def _retained_run_with_paths(run: dict[str, Any] | None) -> dict[str, Any] | None: + if run is None: + return None + paths = retained_run_report_paths(run) + run_dir = Path(run["run_dir"]).resolve() + safe_paths = {} + for key, value in paths.items(): + path = Path(value).resolve() + try: + path.relative_to(run_dir) + except ValueError as exc: + raise HttpProblem( + 400, + f"retained run report path escapes run directory: {value}", + ) from exc + safe_paths[key] = str(path) + return { + **run, + "paths": dict(sorted(safe_paths.items())), + } + + +def _safe_run_dir(runs_dir: Path, run: dict[str, Any]) -> Path: + run_dir_value = run.get("run_dir") + if not isinstance(run_dir_value, str) or not run_dir_value: + raise HttpProblem(400, "retained run is missing run_dir") + run_dir = Path(run_dir_value).resolve() + try: + run_dir.relative_to(runs_dir.resolve()) + except ValueError as exc: + raise HttpProblem(400, f"retained run escapes runs_dir: {run_dir}") from exc + return run_dir + + +def _safe_run_ref(run_dir: Path, ref: Any) -> Path: + if not isinstance(ref, str) or not ref: + raise HttpProblem(400, "artifact manifest entry path must be a non-empty string") + path = (run_dir / ref).resolve() + try: + path.relative_to(run_dir.resolve()) + except ValueError as exc: + raise HttpProblem(400, f"artifact path escapes run directory: {ref}") from exc + return path + + def _display_path(root: Path, path: Path) -> str: try: return str(path.resolve().relative_to(root.resolve())) diff --git a/tests/test_core.py b/tests/test_core.py index ce79b2c..7ac412d 100644 --- a/tests/test_core.py +++ b/tests/test_core.py @@ -7,6 +7,7 @@ import time import unittest from tempfile import TemporaryDirectory from pathlib import Path +from urllib.parse import quote from guide_board.discovery import discover_extensions from guide_board.errors import ValidationError @@ -458,6 +459,64 @@ class CoreArchitectureTests(unittest.TestCase): reports["assessment_package"]["json"]["run_id"], status["result"]["run_id"], ) + self.assertEqual( + reports["submission_package"]["json"]["run_id"], + status["result"]["run_id"], + ) + finally: + service.stop() + + def test_service_exposes_retained_runs_after_restart(self) -> None: + with TemporaryDirectory() as temporary_directory: + runs_dir = Path(temporary_directory) / "runs" + result = run_assessment( + ROOT, + ROOT / "profiles" / "targets" / "sample-repository.json", + ROOT / "profiles" / "assessments" / "sample-noop.json", + runs_dir / "sample", + ) + _write_unsafe_artifact_run(runs_dir / "unsafe-run") + + service = start_service(ROOT, host="127.0.0.1", port=0) + try: + query = f"runs_dir={quote(str(runs_dir), safe='')}" + listing = _request_json(service, "GET", f"/retained-runs?{query}") + self.assertEqual(listing["runs_dir"], str(runs_dir)) + self.assertIn(result["run_id"], [run["run_id"] for run in listing["runs"]]) + + latest = _request_json( + service, + "GET", + f"/retained-runs/latest?{query}&target=sample-repository&assessment=sample-noop-assessment", + ) + self.assertEqual(latest["run"]["run_id"], result["run_id"]) + self.assertIn("submission_package", latest["run"]["paths"]) + + reports = _request_json( + service, + "GET", + f"/retained-runs/{result['run_id']}/reports?{query}", + ) + self.assertEqual( + reports["run"]["paths"]["assessment_package"], + str(runs_dir / "sample" / "reports" / "assessment-package.json"), + ) + + artifacts = _request_json( + service, + "GET", + f"/retained-runs/{result['run_id']}/artifact-manifest?{query}", + ) + self.assertEqual(artifacts["artifact_manifest"], []) + self.assertEqual(artifacts["compatibility"], "current") + + unsafe = _request_json( + service, + "GET", + f"/retained-runs/unsafe-run/artifact-manifest?{query}", + expected_status=400, + ) + self.assertIn("escapes run directory", unsafe["error"]["message"]) finally: service.stop() @@ -603,6 +662,34 @@ def _write_retention_summary( ) +def _write_unsafe_artifact_run(run_dir: Path) -> None: + _write_retention_summary( + run_dir, + "unsafe-run", + "2026-05-07T12:00:00+00:00", + "completed", + {"pass": 1}, + 0, + 1, + ) + reports_dir = run_dir / "reports" + reports_dir.mkdir(parents=True, exist_ok=True) + (reports_dir / "assessment-package.json").write_text( + json.dumps( + { + "artifact_manifest": [ + { + "id": "artifact:unsafe", + "path": "../outside.txt", + "checksum": "sha256:unsafe", + } + ] + } + ), + encoding="utf-8", + ) + + def _request_json( service: ServiceHandle, method: str, diff --git a/workplans/GUIDE-BOARD-WP-0006-service-artifact-access-and-durable-run-index.md b/workplans/GUIDE-BOARD-WP-0006-service-artifact-access-and-durable-run-index.md index 33c01d6..0a6d7c1 100644 --- a/workplans/GUIDE-BOARD-WP-0006-service-artifact-access-and-durable-run-index.md +++ b/workplans/GUIDE-BOARD-WP-0006-service-artifact-access-and-durable-run-index.md @@ -4,12 +4,12 @@ type: workplan title: "Service Artifact Access And Durable Run Index" repo: guide-board domain: markitect -status: active +status: completed owner: codex planning_priority: medium planning_order: 6 created: "2026-05-15" -updated: "2026-05-15" +updated: "2026-05-16" state_hub_workstream_id: "ba008283-1631-467b-868e-1052c3870ab9" --- @@ -40,7 +40,7 @@ existing run artifacts. ```task id: GUIDE-BOARD-WP-0006-T001 -status: todo +status: done priority: high state_hub_task_id: "4d392fc5-6a1c-46f7-9cbf-6c02bbd744c6" ``` @@ -54,11 +54,22 @@ Acceptance: directory layout. - Document the operational tradeoff and failure modes. +Decision: + +- Keep the durable index as retained run summaries and helper scans. +- Do not add a separate service index file for the baseline. + +Progress: + +- Documented reconstruction from `retention-summary.json` files. +- Kept compatibility with older runs that lack newer assessment package or + submission manifest files. + ## D6.2 - Service Run History And Artifact Endpoints ```task id: GUIDE-BOARD-WP-0006-T002 -status: todo +status: done priority: high state_hub_task_id: "8f209920-6b14-4d6f-bfa1-8f1d03bcdbf1" ``` @@ -71,11 +82,21 @@ Acceptance: - Avoid serving arbitrary filesystem paths outside configured run directories. - Add tests for successful retrieval and path-safety failures. +Progress: + +- Added `GET /retained-runs`. +- Added `GET /retained-runs/latest`. +- Added `GET /retained-runs/{run_id}/reports`. +- Added `GET /retained-runs/{run_id}/artifact-manifest`. +- Added path containment checks for report refs and artifact manifest paths. +- Added service tests for retained history retrieval after a fresh service + process and unsafe artifact path rejection. + ## D6.3 - Restart Recovery And Compatibility ```task id: GUIDE-BOARD-WP-0006-T003 -status: todo +status: done priority: medium state_hub_task_id: "0857e7d8-3d23-4426-b7fa-73362d7041a0" ``` @@ -89,11 +110,19 @@ Acceptance: files. - Update service durability documentation with examples. +Progress: + +- Preserved `/runs` as in-memory job history. +- Exposed durable run results through retained-run endpoints after restart. +- Returned a compatibility marker when an older retained run lacks an + assessment package artifact manifest. +- Updated service durability and local API docs. + ## D6.4 - Container And Service Acceptance Tests ```task id: GUIDE-BOARD-WP-0006-T004 -status: todo +status: done priority: medium state_hub_task_id: "900a70fa-65ff-4815-9c0c-31f0da4019f0" ``` @@ -106,6 +135,13 @@ Acceptance: - Document service endpoint usage in local and container modes. - Keep tests dependency-light. +Progress: + +- Added dependency-light service tests for durable run lookup, report paths, and + artifact manifest retrieval. +- Updated container smoke artifact expectations for current run outputs. +- Documented retained-run endpoint usage in local and container modes. + ## Definition Of Done - The local service can expose retained runs and artifacts after restart.