coulomb/the-custodian
Archived
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(sbom): CUST-WP-0013 — expand SBOM infra to terraform, ansible, and tool manifests

Browse Source 
- Migration d6e7f8a9b0c1: add terraform, ansible, tool to Ecosystem enum
- ingest_sbom.py: new Ansible Galaxy requirements.yml parser (collections + roles)
- ingest_sbom.py: new sbom-tools.yaml manifest parser (agent-generated tool deps)
- ingest_sbom.py: promote .terraform.lock.hcl parser from ecosystem=other → terraform
- ingest_sbom.py: detect_all() runs all four parsers in one comprehensive scan
- capture_sbom_tools.py: agent-assisted tool manifest generator (claude -p)
- prompts/sbom-capture-agent.md: parameterised prompt for repo tool discovery
- Makefile: capture-tools target; ingest-sbom updated docs and DRY_RUN support
- 29 unit tests covering all new parsers and detect_all() behaviour
- canon/standards/sbom-convention_v0.1.md: updated with four-mechanism model and workflow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-03-12 04:40:26 +01:00
parent 3c69ad2929
commit 1c94f5545c
9 changed files with 1378 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files

101
canon/standards/sbom-convention_v0.1.md
View File

@@ -6,7 +6,7 @@ domain: custodian
status: active
version: "0.1"
created: "2026-03-01"
updated: "2026-03-01"
updated: "2026-03-12"
---
# SBOM Convention v0.1 — Dependency Tracking & Licence Governance
@@ -27,20 +27,23 @@ dashboard (`/sbom`) provides domain-level and repo-level drill-down.
---
## 1. Authoritative Lockfiles per Ecosystem
## 1. Capture Mechanisms
| Ecosystem | Authoritative file | Notes |
|-----------|-------------------|-------|
| Python | `uv.lock` | Preferred. `requirements.txt` accepted as fallback |
| Node / npm | `package-lock.json` | Preferred. `yarn.lock` accepted |
| Rust | `Cargo.lock` | Auto-detected |
| Terraform | `.terraform.lock.hcl` | Provider pins; ecosystem stored as `other` until ENUM extended |
| Go | `go.sum` | *Not yet parsed — planned* |
| Java / JVM | `gradle.lockfile` / `pom.xml` | *Not yet parsed — planned* |
| Ansible | `requirements.yml` | *Not yet parsed — planned* |
`ingest_sbom.py` runs all four mechanisms in a single scan when given `--repo-path`.
No flags needed — comprehensive detection is the default.
**Principle:** commit lockfiles to the repo. Lockfiles are the SBOM source
of truth; do not generate them at ingest time.
| Mechanism | File(s) | Ecosystem | Detection scope |
|-----------|---------|-----------|-----------------|
| **Package manager lockfiles** | `uv.lock`, `requirements.txt`, `package-lock.json`, `yarn.lock`, `Cargo.lock` | `python`, `node`, `rust` | Anywhere in tree |
| **Terraform provider lock** | `.terraform.lock.hcl` | `terraform` | Anywhere in tree |
| **Ansible Galaxy manifest** | `ansible/requirements.yml` or `.yaml` | `ansible` | Under directories named `ansible/` |
| **Tool manifest** | `sbom-tools.yaml` (repo root) | `tool`, `ansible`, `terraform`, etc. | Repo root only |
**Go / Java parsers** (`go.sum`, `pom.xml`, `gradle.lockfile`) are *not yet
implemented* — planned for a future workplan.
**Principle:** commit lockfiles and `sbom-tools.yaml` to the repo. These are
the SBOM source of truth; do not generate them at ingest time.
---
@@ -64,27 +67,35 @@ curl -s http://127.0.0.1:8000/repos/ | python3 -m json.tool
## 3. SBOM Ingestion
### 3.1 Standard ingest (single lockfile at repo root)
### 3.1 Standard ingest (all mechanisms, recommended)
```bash
cd ~/the-custodian/state-hub
make ingest-sbom REPO=<slug> REPO_PATH=/path/to/repo
```
The script auto-detects the first recognised lockfile at `REPO_PATH`.
`ingest_sbom.py` automatically runs all four mechanisms in one scan — lockfiles,
Terraform provider locks, Ansible Galaxy manifests, and `sbom-tools.yaml`. All
results are merged into a single snapshot. Non-dep directories (`.venv`,
`node_modules`, `.git`, `dist`, etc.) are automatically skipped.
### 3.2 Multi-ecosystem repos (recommended for complex repos)
### 3.2 Repos with system-level tools: capture first, then ingest
Use `SCAN=1` to walk the repo tree and combine **all** lockfiles into a single
snapshot. Non-dep directories (`.venv`, `node_modules`, `.git`, `dist`, etc.)
are automatically skipped.
For repos that use system-level tools not tracked by any lockfile (Terraform
binary, Helm, kubectl, k3s, goss, etc.):
```bash
make ingest-sbom REPO=the-custodian SCAN=1 REPO_PATH=/home/worsch/the-custodian
```
# Step 1: generate sbom-tools.yaml via agent
make capture-tools REPO=<slug> REPO_PATH=/path/to/repo
This is the correct approach for repos that contain both a backend and a
frontend (e.g., a Python API + Node/Observable dashboard).
# Step 2: review sbom-tools.yaml — correct any confidence: low entries
# Step 3: commit sbom-tools.yaml
git -C /path/to/repo add sbom-tools.yaml && git -C /path/to/repo commit -m "chore(sbom): add tool manifest"
# Step 4: ingest everything
make ingest-sbom REPO=<slug> REPO_PATH=/path/to/repo
```
### 3.3 Explicit lockfile path
@@ -96,8 +107,7 @@ Multiple lockfiles can be passed by calling the script directly with repeated
`--lockfile` flags:
```bash
cd ~/the-custodian/state-hub
.venv/bin/python scripts/ingest_sbom.py \
uv run python scripts/ingest_sbom.py \
--repo <slug> \
--lockfile /path/to/uv.lock \
--lockfile /path/to/package-lock.json
@@ -106,11 +116,40 @@ cd ~/the-custodian/state-hub
### 3.4 Dry run (inspect without submitting)
```bash
make ingest-sbom REPO=<slug> SCAN=1 REPO_PATH=/path/to/repo
# append: add --dry-run to the command, or run the script directly:
.venv/bin/python scripts/ingest_sbom.py --repo <slug> --scan --repo-path /path/to/repo --dry-run
make ingest-sbom REPO=<slug> REPO_PATH=/path/to/repo DRY_RUN=1
```
### 3.5 sbom-tools.yaml: the tool manifest
Create `sbom-tools.yaml` at the repo root for any system-level tools not
covered by lockfiles. Schema:
```yaml
# sbom-tools.yaml
tools:
- name: terraform
version: "1.9.5" # confidence: medium
ecosystem: terraform
license_spdx: BSL-1.1
is_direct: true
is_dev: false
- name: helm
version: null # confidence: low (no version pin found)
ecosystem: tool
license_spdx: Apache-2.0
is_direct: true
is_dev: false
```
**Valid ecosystem values:** `python`, `node`, `rust`, `go`, `java`, `terraform`,
`ansible`, `tool`, `other`
Annotate each version with a `# confidence: high/medium/low` comment.
Entries with `confidence: low` need human verification before committing.
The `make capture-tools` command generates this file automatically using the
SBOM capture agent prompt (`state-hub/prompts/sbom-capture-agent.md`).
---
## 4. Snapshot Semantics
@@ -248,10 +287,14 @@ The SBOM dashboard aggregates across all repos within a domain in the
## 10. Planned Enhancements
- **Go / Java parsers** — add to `ingest_sbom.py`
- **Go / Java parsers** — add `go.sum`, `pom.xml`, `gradle.lockfile` support to `ingest_sbom.py`
- **Versioned snapshots** — retain history per repo for trend analysis
- **Licence override file** — allow repos to document known-acceptable
copyleft exceptions (`.sbom-overrides.yaml`)
- **CI integration** — GitHub Actions step to run ingest on lockfile change
- **Direct-dep detection for uv.lock** — parse `pyproject.toml` `[project.dependencies]`
to mark direct deps accurately
- **Galaxy API licence lookup** — resolve `license_spdx` for Ansible collections
via the Galaxy API at ingest time
- **Tool version pinning guidance** — tooling to detect `confidence: low` entries
across all registered repos and flag them for resolution