3.1 KiB
INTENT
Purpose
This repository exists to provide a provider-neutral interface for interacting with large language models (LLMs) in Python.
It ensures that applications can use LLM capabilities without being tightly coupled to any specific provider, API, or execution environment.
Primary Utility
The repository provides a unified adapter layer that:
- Abstracts over multiple LLM providers and execution modes
- Standardizes request, response, and configuration handling
- Enables interchangeable use of hosted APIs and local tooling (e.g. CLI-based models)
- Supports embeddings, token estimation, and related primitives
- Enables dynamic utility by cost optimizations
It transforms heterogeneous LLM ecosystems into a consistent, composable programming interface.
Intended Users
- Application developers integrating LLM capabilities into their systems
- Library and framework authors requiring provider-agnostic LLM primitives
- Automation systems (
atm) orchestrating LLM-assisted workflows - LLM agents (
agt) operating across different model providers
Strategic Role in the System
This repository acts as the LLM abstraction layer within the broader system:
- It decouples application logic from provider-specific implementations
- It enables runtime flexibility and provider switching without code changes
- It supports architectures where LLM usage is optional, replaceable, and testable
It allows higher-level systems to treat LLMs as pluggable capabilities rather than fixed dependencies.
Strategic Boundaries
This repository is not intended to:
- Provide application-level agent frameworks or workflows
- Define prompting strategies, routing policies, or domain-specific logic
- Manage secrets, credentials, or organizational access policies
- Own or implement LLM providers themselves
Its responsibility is limited to clean abstraction and integration of LLM capabilities.
Design Principles
-
Abstraction over providers Consumers depend on a stable adapter interface, not on vendor APIs
-
Composability LLM functionality should be usable as a building block in larger systems
-
Replaceability Providers and execution modes must be interchangeable without affecting consumers
-
Deterministic integration boundaries Non-LLM logic must remain testable and independent of LLM variability
-
Minimal opinionation The library provides primitives, not policies
Maturity Target
A mature version of this repository should:
- Provide a stable, versioned core adapter contract for LLM interaction
- Support a broad range of providers and execution environments
- Enable seamless switching and fallback between providers
- Offer consistent handling of responses, errors, and usage metrics
- Serve as the default integration layer for LLM capabilities across dependent systems
Stability Note
Changes to this file represent a deliberate shift in the abstraction boundaries or role of this repository.
Such changes should be rare, as they affect all downstream systems relying on provider-neutral LLM integration.