Open-source AI gateway and Python SDK for routing LLM calls through a unified OpenAI-compatible interface.
Open the source and read safety notes before installing.
Source-backed facts for citing this resource, derived directly from the registry — also available as plain text for AI assistants.
Decision playbook
Signals are present but mixed. Use the checklist below to confirm the source and operational safety for your environment.
0
78
—
No baseline selected
No major trust-signal divergence detected in the current selection.
Confirm ownership and provenance before trusting install instructions.
Source link availableRequired
Open the canonical repository and verify ownership.
Source provenance statusRequired
Marked as source-backed.
Metadata reviewed
Registry metadata indicates a reviewed listing.
Validate risk disclosures before installation or API wiring.
Safety notes presentRequired
Review the listed safety guidance before running commands.
Privacy notes presentRequired
Review data handling notes before connecting accounts or secrets.
Trust level risk gateRequired
Trust level does not block evaluation.
Check package metadata and artifact integrity signals.
Install payload available
Install or copy payload is available for review.
Package verification flag
No package verification flag provided.
Checksum metadata
No checksum provided for downloaded artifact.
Use compare context to validate trade-offs before adoption.
Compare tray has multiple entries
Add at least one more entry to compare trust differences.
Baseline comparison available
No baseline peer selected yet.
Diverging trust signals identified
No major trust-signal divergence found.
Setup at a glance
Copy-ready — paste the snippet to get started.
Adoption plan
Current risk score 16/100. Use staged verification before broader rollout.
Validate source and review signals before any execution.
Confirm source provenanceRequired
Source URL/provenance metadata is present.
Confirm metadata review state
Listing has review metadata.
Verify install payload
Install/config payload exists and can be inspected.
Confirm safety, privacy, and package integrity signals.
Review safety notesRequired
Safety notes are present.
Review privacy notesRequired
Privacy notes are present.
Verify package integrity metadata
No package verification/checksum metadata.
Adopt in controlled steps based on the selected plan.
Run in isolated sandbox firstRequired
Use a constrained sandbox and observe behavior across multiple tasks.
Roll out graduallyRequired
Roll out to a small cohort before wider usage.
Set monitoring and fallback
Define rollback path and monitor errors after adoption.
Evidence readiness
Required evidence gates are covered (5/6 signals complete).
Source repository/provenance is listed.
Required in this preset
Review metadata is present.
Required in this preset
Safety notes are present.
Required in this preset
Privacy notes are present.
Optional in this preset
Package integrity metadata is missing.
Optional in this preset
Install payload is available.
Required in this preset
Required evidence gates are covered for this preset.
Decision timeline
5/6 steps complete with no blocking gaps for this preset.
triage
Source/provenance metadata is available.
triage
Review metadata is available.
verify
Safety notes are available.
verify
Privacy notes are available.
verify
Package integrity metadata is missing.
rollout
Install payload is available.
No required blockers for this timeline preset.
Prerequisite readiness
3 prerequisites to line up before setup. Have accounts and credentials ready first. Includes a review or approval gate.
Safety & privacy surface
3 safety and 3 privacy notes across 3 risk areas. Review closely: credentials & tokens, network access, third-party handling.
Disclosure: editorial
## Editorial notes
LiteLLM is a strong fit for Claude and agent workflows that need one OpenAI-compatible gateway in front of multiple model providers. The official docs describe both the Python SDK and the self-hosted proxy, including routing, load balancing, virtual keys, spend tracking, guardrails, observability, and an admin UI.
## Source notes
- The official documentation describes LiteLLM as an open-source library with a unified interface for 100+ LLMs and a self-hosted LLM gateway/proxy.
- The GitHub README describes LiteLLM as an open-source AI gateway for 100+ LLMs with OpenAI-format calls.
- The PyPI package is published as `litellm` for Python SDK and proxy installation paths.
## Duplicate check
Checked current `content/tools/`, open pull requests, and the repository for `LiteLLM`, `BerriAI`, `llm gateway`, `model router`, and `openai compatible proxy`. No existing LiteLLM listing or open duplicate PR was found.
## Disclosure
Editorial listing. No paid placement or affiliate link is used.LiteLLM is a strong fit for Claude and agent workflows that need one OpenAI-compatible gateway in front of multiple model providers. The official docs describe both the Python SDK and the self-hosted proxy, including routing, load balancing, virtual keys, spend tracking, guardrails, observability, and an admin UI.
litellm for Python SDK and proxy installation paths.Checked current content/tools/, open pull requests, and the repository for LiteLLM, BerriAI, llm gateway, model router, and openai compatible proxy. No existing LiteLLM listing or open duplicate PR was found.
Editorial listing. No paid placement or affiliate link is used.
Show that LiteLLM is listed on HeyClaude. Paste this Markdown into your README — it renders the badge and links back to this page.
[](https://heyclau.de/entry/tools/litellm)LiteLLM side by side with 3 alternatives on trust, install, platform support, and disclosed safety notes — all from reviewed registry metadata.
1 trust signal differ across this comparison (Submitter).
| Field | Open-source AI gateway and Python SDK for routing LLM calls through a unified OpenAI-compatible interface. Open dossier | Open-source AI gateway from Portkey for routing to 1600+ LLMs through one OpenAI-compatible API, with automatic retries, fallbacks, load balancing, conditional routing, guardrails, caching, and observability, self-hostable via npx, Docker, or edge deployments. Open dossier | Open-source, self-hostable AI engine that runs LLMs, vision, voice, image, and video models on your own hardware behind one API, with drop-in OpenAI, Anthropic, and ElevenLabs API compatibility, composable on-demand backends, and no GPU required. Open dossier | Local model runner for downloading, serving, and integrating open models with developer tools and agent workflows. Open dossier |
|---|---|---|---|---|
| Next steps | ||||
| Trust | ||||
| Review status | ReviewedMaintainer reviewed | ReviewedMaintainer reviewed | ReviewedMaintainer reviewed | ReviewedMaintainer reviewed |
| Package trust | Package not verified | Package not verified | Package not verified | Package not verified |
| Source provenance | Source-backed | Source-backed | Source-backed | Source-backed |
| SubmitterDiffers | oktofeesh1 | davion-knight | davion-knight | oktofeesh1 |
| Install risk | Review first | Review first | Review first | Review first |
| Notes | Safety ✓ Privacy ✓ | Safety ✓ Privacy ✓ | Safety ✓ Privacy ✓ | Safety ✓ Privacy ✓ |
| Brand | ||||
| Category | tools | tools | tools | tools |
| Source | Source-backed | Source-backed | Source-backed | Source-backed |
| Author | BerriAI | Portkey-AI | mudler | Ollama |
| Added | 2026-06-03 | 2026-07-09 | 2026-07-10 | 2026-06-03 |
| Platforms | ||||
| Harness | ||||
| Source repo | — | — | — | — |
| Safety notes | ✓LiteLLM can proxy requests to multiple model providers, so route and fallback behavior should be reviewed before production use. Gateway deployments can expose model access to teams or applications; configure authentication, budgets, rate limits, and network access intentionally. Avoid logging sensitive prompt, response, or credential material when enabling debugging, observability, or admin features. | ✓The gateway sits in the path of your LLM traffic and forwards requests to configured providers using the API keys you supply, so scope those provider keys to the minimum needed and store them securely. Self-hosting exposes a local or deployed endpoint; run it on a trusted network or behind authentication, and do not expose an unauthenticated gateway to the public internet. Guardrails, retries, fallbacks, and conditional routing change how and where requests are sent, so review the gateway configuration before using it for production traffic. Caching returns stored responses for matching requests; confirm that cached content is appropriate to reuse before enabling it for sensitive or per-user data. Treat provider responses returned through the gateway as untrusted input for any downstream action, and keep production configuration narrower than local examples. | ✓LocalAI runs a server that exposes an API; run it on a trusted network or behind authentication, and do not expose an unauthenticated endpoint on a public interface. It uses API-key auth, user quotas, and role-based access for multi-user setups; enable and scope these before sharing an instance. Backends are pulled on demand and run model code locally; pull backends and models from sources you trust, and verify model licenses before serving them. Treat model outputs as untrusted input for any downstream action, and keep production configuration and exposed ports narrower than local quickstart examples. When installing from a downloaded artifact, follow the project's platform notes and verify the source before running it. | ✓Downloaded models can be large and may carry their own license, usage, and safety constraints; review model cards before use. Ollama exposes a local service and REST API, so bind addresses, firewall rules, and shared-machine access should be configured intentionally. Generated outputs from local models still need review before they are applied to code, documentation, or operational decisions. |
| Privacy notes | ✓Prompts and responses pass through the LiteLLM process and then to the selected upstream model provider. Gateway logs, spend tracking, and observability integrations may retain request metadata or payload excerpts depending on configuration. Self-hosted deployments still depend on the privacy terms of each configured model provider. | ✓Requests routed through the gateway, including prompts and inputs, are forwarded to the configured model providers, which process that data under their own terms. If observability or logging is enabled, prompts, responses, metadata, and metrics can be recorded in the store you configure, so choose a destination and retention policy deliberately. Caching stores request and response data to serve repeated calls; scope and expire the cache appropriately for the sensitivity of the traffic. Provider API keys, gateway configuration, and any request logs should be kept out of version control and access-controlled like other secrets and operational data. | ✓Running LocalAI keeps inference on your own hardware, so prompts and data do not leave your environment unless you configure it to call external services. Requests, prompts, and generated outputs can be logged depending on your configuration; choose logging and retention settings deliberately, especially for sensitive data. Served models and any stored inputs or outputs should be kept with appropriate access controls, particularly on multi-user instances. If you connect LocalAI to external providers or expose it to other services, apply normal credential hygiene and keep configuration out of version control. | ✓Local prompts and responses can stay on the machine when using local models, but they may appear in client logs, shell history, or application telemetry around the integration. Any remote model source, community integration, or connected chat/workflow client may add its own data handling behavior. Do not assume local execution removes the need to protect secrets or sensitive repository context from prompts and logs. |
| Prerequisites |
|
|
|
|
| Install | — | — | — | — |
| Config | — | — | — | — |
| Citations | ||||
| Claim | Unclaimed | Unclaimed | Unclaimed | Unclaimed |
Open-source AI gateway to 1600+ LLMs with retries, fallbacks, routing, guardrails, caching, and observability.
Source-backed guides for putting this to work.
Loading live community signals…
A short, calm digest of reviewed Claude resources. Unsubscribe any time.