tegwick
c7a893f068
Introduces TPSC for tracking external service dependencies with GDPR
compliance maturity (CNIL/IAPP CMMI scale), pricing model, ToS, and
data retention information across all repos.
Primary data:
- canon/tpsc/{openai,anthropic,gemini,openrouter}-api.yaml — service definitions
- tpsc.yaml in each repo (llm-connect seeded with 4 services)
State-hub additions:
- Migration j7e8f9a0b1c2: tpsc_catalog + tpsc_snapshots + tpsc_entries
- api/models/tpsc.py, api/schemas/tpsc.py, api/routers/tpsc.py
- /tpsc/catalog/, /tpsc/ingest/, /tpsc/snapshots/, /tpsc/report/gdpr endpoints
- 4 MCP tools: register_service, list_services, ingest_tpsc_tool, get_gdpr_report
- scripts/ingest_tpsc.py + make ingest-tpsc[/-all] targets
- Dashboard: tpsc.md page + docs/tpsc.md
GDPR maturity scale: unknown | non_compliant | initial | developing | defined | managed | certified
Warnings triggered at: unknown, non_compliant, initial
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
211 lines
5.2 KiB
Markdown
211 lines
5.2 KiB
Markdown
---
|
|
id: CUST-WP-0023
|
|
type: feature
|
|
title: Third-Party Services Catalog (TPSC)
|
|
domain: custodian
|
|
status: done
|
|
owner: custodian-agent
|
|
topic_slug: custodian
|
|
created: 2026-03-19
|
|
updated: 2026-03-20
|
|
state_hub_workstream_id: "23208a99-0ef6-4154-9454-a2f2065b6b19"
|
|
---
|
|
|
|
# TPSC — Third-Party Services Catalog
|
|
|
|
Track external service dependencies (APIs, SaaS, CLIs) across all repos.
|
|
Primary data lives in repos (`tpsc.yaml`) and a canon catalog
|
|
(`canon/tpsc/<slug>.yaml`). State-hub collects and reports.
|
|
|
|
GDPR maturity scale (CNIL/IAPP CMMI-aligned):
|
|
`unknown | non_compliant | initial | developing | defined | managed | certified`
|
|
|
|
Pricing model: `free | paid | freemium | usage_based | unknown`
|
|
|
|
## Task: DB migration — tpsc_catalog, tpsc_snapshots, tpsc_entries
|
|
|
|
```task
|
|
id: CUST-WP-0023-T01
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "038a0284-bb76-4ce7-8861-1686667acbb5"
|
|
```
|
|
|
|
Create Alembic migration `j7e8f9a0b1c2_tpsc.py`.
|
|
|
|
Tables:
|
|
- `tpsc_catalog`: id, slug (unique), name, provider, category, website_url,
|
|
pricing_model, gdpr_maturity, gdpr_notes, dpa_available, tos_url,
|
|
privacy_policy_url, data_processing_regions (JSON), data_retention_notes,
|
|
status (active/deprecated), created_at, updated_at
|
|
- `tpsc_snapshots`: id, repo_id (FK managed_repos nullable), snapshot_at,
|
|
source_file, entry_count
|
|
- `tpsc_entries`: id, snapshot_id (FK), catalog_id (FK tpsc_catalog nullable),
|
|
service_slug, purpose, auth_type, endpoint_override, notes
|
|
|
|
---
|
|
|
|
## Task: SQLAlchemy models
|
|
|
|
```task
|
|
id: CUST-WP-0023-T02
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "990d4e58-35b0-45f0-8de6-8955049aa7d5"
|
|
```
|
|
|
|
Create `api/models/tpsc.py` with TPSCCatalog, TPSCSnapshot, TPSCEntry models.
|
|
Register in `api/models/__init__.py`.
|
|
|
|
---
|
|
|
|
## Task: Pydantic schemas
|
|
|
|
```task
|
|
id: CUST-WP-0023-T03
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "5feeb161-b654-4e4a-8a6a-bb685c239ce5"
|
|
```
|
|
|
|
Create `api/schemas/tpsc.py` with Read/Create schemas for all three models.
|
|
Include `GDPRMaturity` and `PricingModel` string enums.
|
|
`TPSCCatalog` schema: include `gdpr_warning: bool` computed field
|
|
(True when gdpr_maturity in [unknown, non_compliant, initial]).
|
|
|
|
---
|
|
|
|
## Task: FastAPI router /tpsc/
|
|
|
|
```task
|
|
id: CUST-WP-0023-T04
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "593471b4-cd3a-4251-8c5c-ee42b6a9e089"
|
|
```
|
|
|
|
Create `api/routers/tpsc.py`:
|
|
- `GET /tpsc/catalog/` — list services (filter: gdpr_maturity, category, pricing_model)
|
|
- `GET /tpsc/catalog/{slug}` — single service
|
|
- `POST /tpsc/catalog/` — register/upsert service
|
|
- `POST /tpsc/ingest/` — accept snapshot + entries for a repo
|
|
- `GET /tpsc/snapshots/` — list snapshots (filter: repo_slug)
|
|
- `GET /tpsc/report/gdpr` — aggregated GDPR warnings across all repos
|
|
|
|
Register in `api/main.py`.
|
|
|
|
---
|
|
|
|
## Task: MCP tools
|
|
|
|
```task
|
|
id: CUST-WP-0023-T05
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "7370d020-06cf-4ebe-9f78-619e41c4b85c"
|
|
```
|
|
|
|
Add to `mcp_server/server.py`:
|
|
- `register_service(slug, name, provider, pricing_model, gdpr_maturity, ...)`
|
|
- `list_services(gdpr_maturity?, category?, pricing_model?)`
|
|
- `ingest_tpsc_tool(repo_slug)` — runs ingest_tpsc.py for the repo
|
|
- `get_gdpr_report()` — returns warning summary across all repos
|
|
|
|
---
|
|
|
|
## Task: Ingest script
|
|
|
|
```task
|
|
id: CUST-WP-0023-T06
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "5ec305d3-fdfb-4f81-b3ab-1ea57a4ec0c5"
|
|
```
|
|
|
|
Create `scripts/ingest_tpsc.py`:
|
|
- Reads `tpsc.yaml` from repo root (auto-detected via registered local_path)
|
|
- Resolves catalog_id by slug for each entry
|
|
- POSTs snapshot + entries to `/tpsc/ingest/`
|
|
- `--dry-run` flag
|
|
- `--repo SLUG` or `--all` flags
|
|
|
|
Add Makefile targets:
|
|
```
|
|
make ingest-tpsc REPO=<slug>
|
|
make ingest-tpsc-all
|
|
```
|
|
|
|
---
|
|
|
|
## Task: Canon service catalog seed files
|
|
|
|
```task
|
|
id: CUST-WP-0023-T07
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "21c13b4d-585a-4a60-a283-29baa2dd6d7d"
|
|
```
|
|
|
|
Create `canon/tpsc/` with YAML files for:
|
|
- `openai-api.yaml`
|
|
- `anthropic-api.yaml`
|
|
- `gemini-api.yaml`
|
|
- `openrouter-api.yaml`
|
|
|
|
Each file: slug, name, provider, category, pricing_model, gdpr_maturity,
|
|
gdpr_notes, dpa_available, tos_url, privacy_policy_url,
|
|
data_processing_regions, data_retention_notes.
|
|
|
|
---
|
|
|
|
## Task: tpsc.yaml for llm-connect
|
|
|
|
```task
|
|
id: CUST-WP-0023-T08
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "d658f81b-7408-456d-b4bd-440b44a67e43"
|
|
```
|
|
|
|
Create `/home/worsch/llm-connect/tpsc.yaml` declaring:
|
|
openai-api, anthropic-api (via claude_code CLI), gemini-api, openrouter-api.
|
|
|
|
---
|
|
|
|
## Task: Dashboard page
|
|
|
|
```task
|
|
id: CUST-WP-0023-T09
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "a5367fc4-ef12-4f52-b642-d33b91b7cc2c"
|
|
```
|
|
|
|
Create `dashboard/src/tpsc.md`:
|
|
- Service catalog table with GDPR maturity color coding
|
|
(red: non_compliant/unknown, amber: initial/developing, green: defined+)
|
|
- Warning cards for repos using non-green services
|
|
- Per-repo breakdown table
|
|
- Pricing model summary
|
|
|
|
Add to `observablehq.config.js` nav.
|
|
|
|
---
|
|
|
|
## Task: Documentation page
|
|
|
|
```task
|
|
id: CUST-WP-0023-T10
|
|
status: done
|
|
priority: low
|
|
state_hub_task_id: "6cf1aaec-5c24-4cb5-a1aa-415caeaaca10"
|
|
```
|
|
|
|
Create `dashboard/src/docs/tpsc.md` explaining:
|
|
- TPSC concept and rationale
|
|
- tpsc.yaml format with full example
|
|
- GDPR maturity scale definitions (linked to CNIL/IAPP)
|
|
- How to add a new service to the canon catalog
|
|
|
|
Add to docs nav in `observablehq.config.js`.
|