Open-source AI gateway from Portkey for routing to 1600+ LLMs through one OpenAI-compatible API, with automatic retries, fallbacks, load balancing, conditional routing, guardrails, caching, and observability, self-hostable via npx, Docker, or edge deployments.
Open the source and read safety notes before installing.
Source-backed facts for citing this resource, derived directly from the registry — also available as plain text for AI assistants.
Decision playbook
Signals are present but mixed. Use the checklist below to confirm the source and operational safety for your environment.
0
78
—
No baseline selected
No major trust-signal divergence detected in the current selection.
Confirm ownership and provenance before trusting install instructions.
Source link availableRequired
Open the canonical repository and verify ownership.
Source provenance statusRequired
Marked as source-backed.
Metadata reviewed
Registry metadata indicates a reviewed listing.
Validate risk disclosures before installation or API wiring.
Safety notes presentRequired
Review the listed safety guidance before running commands.
Privacy notes presentRequired
Review data handling notes before connecting accounts or secrets.
Trust level risk gateRequired
Trust level does not block evaluation.
Check package metadata and artifact integrity signals.
Install payload available
Install or copy payload is available for review.
Package verification flag
No package verification flag provided.
Checksum metadata
No checksum provided for downloaded artifact.
Use compare context to validate trade-offs before adoption.
Compare tray has multiple entries
Add at least one more entry to compare trust differences.
Baseline comparison available
No baseline peer selected yet.
Diverging trust signals identified
No major trust-signal divergence found.
Adoption plan
Current risk score 16/100. Use staged verification before broader rollout.
Validate source and review signals before any execution.
Confirm source provenanceRequired
Source URL/provenance metadata is present.
Confirm metadata review state
Listing has review metadata.
Verify install payload
Install/config payload exists and can be inspected.
Confirm safety, privacy, and package integrity signals.
Review safety notesRequired
Safety notes are present.
Review privacy notesRequired
Privacy notes are present.
Verify package integrity metadata
No package verification/checksum metadata.
Adopt in controlled steps based on the selected plan.
Run in isolated sandbox firstRequired
Use a constrained sandbox and observe behavior across multiple tasks.
Roll out graduallyRequired
Roll out to a small cohort before wider usage.
Set monitoring and fallback
Define rollback path and monitor errors after adoption.
Evidence readiness
Required evidence gates are covered (5/6 signals complete).
Source repository/provenance is listed.
Required in this preset
Review metadata is present.
Required in this preset
Safety notes are present.
Required in this preset
Privacy notes are present.
Optional in this preset
Package integrity metadata is missing.
Optional in this preset
Install payload is available.
Required in this preset
Required evidence gates are covered for this preset.
Decision timeline
5/6 steps complete with no blocking gaps for this preset.
triage
Source/provenance metadata is available.
triage
Review metadata is available.
verify
Safety notes are available.
verify
Privacy notes are available.
verify
Package integrity metadata is missing.
rollout
Install payload is available.
No required blockers for this timeline preset.
## Editorial notes
Portkey AI Gateway is useful when Claude-adjacent teams want a single, reliable entry point in front of many model providers instead of wiring provider SDKs, retries, and failover into every application. It is a lightweight, open-source gateway that exposes one OpenAI-compatible API and routes requests to 1600+ language, vision, audio, and image models, so application code can switch or combine providers through configuration rather than code changes.
This is distinct from the agent frameworks and libraries in the directory: rather than defining agents or structured outputs, Portkey AI Gateway is the routing and reliability layer that those workflows send their model calls through. It is self-hostable and can run locally, in a container, or at the edge.
## Key capabilities
- **Unified API** — a single OpenAI-compatible endpoint that fronts many providers, so the same client code targets different models by configuration.
- **Broad provider coverage** — routing to 1600+ language, vision, audio, and image models across many providers.
- **Reliability** — automatic retries and provider fallbacks to prevent downtime when a provider errors or is unavailable.
- **Scaling and routing** — load balancing across providers or keys, and conditional routing to send requests to different targets based on rules.
- **Guardrails** — apply input and output guardrails to model traffic passing through the gateway.
- **Caching** — return stored responses for matching requests to cut latency and cost where reuse is appropriate.
- **Observability** — capture logs, metrics, and traces for the requests flowing through the gateway.
- **Virtual keys and budgets** — manage provider credentials and usage controls centrally (in the broader Portkey product).
- **MCP gateway** — an option to manage MCP servers with enterprise auth and observability.
## How teams use it
- **Provider failover** — keep an AI feature available by falling back to an alternate provider or model when the primary fails.
- **Cost and latency control** — cache repeatable responses and route cheaper or faster models for suitable requests.
- **Multi-provider experiments** — compare or load-balance across providers behind one API without changing application code.
- **Policy enforcement** — apply guardrails and routing rules to every model call from a central place.
- **Operational visibility** — collect logs and metrics for AI traffic to debug and monitor usage.
## Running the gateway
The gateway is open source and self-hostable. It can be run locally with `npx @portkey-ai/gateway`, which serves an OpenAI-compatible API and a console on local port 8787, and it can also be deployed with Docker or to edge platforms per the deployment guides. Applications then point an OpenAI-compatible client at the gateway and pass the target provider and model, and the gateway applies the configured routing, retries, fallbacks, guardrails, and caching.
## Source notes
- The official repository describes the AI Gateway as a lightweight, open-source, enterprise-ready solution for fast, reliable, and secure routing to 1600+ language, vision, audio, and image models through one API.
- Documented capabilities include automatic retries and fallbacks, load balancing and conditional routing, guardrails, caching, a unified OpenAI-compatible API, and observability, plus an MCP gateway option for managing MCP servers with auth and observability.
- The gateway runs locally with `npx @portkey-ai/gateway` on local port 8787 and can also be self-hosted via Docker or edge deployments.
- The gateway is published on npm as `@portkey-ai/gateway`.
- The GitHub repository is `Portkey-AI/gateway`, is MIT licensed, and is maintained by Portkey; a managed platform and hosted gateway are available separately from the open-source project.
## Duplicate check
Checked current `content/tools/`, `content/mcp/`, agents, skills, hooks, rules, commands, guides, open pull requests, and repository-wide content for `Portkey`, `portkey`, `portkey.ai`, `github.com/Portkey-AI/gateway`, `@portkey-ai/gateway`, `AI Gateway`, and `LLM gateway`. Existing entries cover model libraries, agent frameworks, and observability tools, but no dedicated Portkey AI Gateway tools entry, Portkey source URL duplicate, or open duplicate PR was found.
## Disclosure
Editorial listing. No paid placement or affiliate link is used.Portkey AI Gateway is useful when Claude-adjacent teams want a single, reliable entry point in front of many model providers instead of wiring provider SDKs, retries, and failover into every application. It is a lightweight, open-source gateway that exposes one OpenAI-compatible API and routes requests to 1600+ language, vision, audio, and image models, so application code can switch or combine providers through configuration rather than code changes.
This is distinct from the agent frameworks and libraries in the directory: rather than defining agents or structured outputs, Portkey AI Gateway is the routing and reliability layer that those workflows send their model calls through. It is self-hostable and can run locally, in a container, or at the edge.
The gateway is open source and self-hostable. It can be run locally with npx @portkey-ai/gateway, which serves an OpenAI-compatible API and a console on local port 8787, and it can also be deployed with Docker or to edge platforms per the deployment guides. Applications then point an OpenAI-compatible client at the gateway and pass the target provider and model, and the gateway applies the configured routing, retries, fallbacks, guardrails, and caching.
npx @portkey-ai/gateway on local port 8787 and can also be self-hosted via Docker or edge deployments.@portkey-ai/gateway.Portkey-AI/gateway, is MIT licensed, and is maintained by Portkey; a managed platform and hosted gateway are available separately from the open-source project.Checked current content/tools/, content/mcp/, agents, skills, hooks, rules, commands, guides, open pull requests, and repository-wide content for Portkey, portkey, portkey.ai, github.com/Portkey-AI/gateway, @portkey-ai/gateway, AI Gateway, and LLM gateway. Existing entries cover model libraries, agent frameworks, and observability tools, but no dedicated Portkey AI Gateway tools entry, Portkey source URL duplicate, or open duplicate PR was found.
Editorial listing. No paid placement or affiliate link is used.
Show that Portkey AI Gateway is listed on HeyClaude. Paste this Markdown into your README — it renders the badge and links back to this page.
[](https://heyclau.de/entry/tools/portkey-ai-gateway)Portkey AI Gateway side by side with 3 alternatives on trust, install, platform support, and disclosed safety notes — all from reviewed registry metadata.
1 trust signal differ across this comparison (Submitter).
Next steps differ across entries — use the actions in the table below to copy install commands and source links per resource.
| Field | Open-source AI gateway from Portkey for routing to 1600+ LLMs through one OpenAI-compatible API, with automatic retries, fallbacks, load balancing, conditional routing, guardrails, caching, and observability, self-hostable via npx, Docker, or edge deployments. Open dossier | Open-source AI gateway and Python SDK for routing LLM calls through a unified OpenAI-compatible interface. Open dossier | Open-source AI engineering platform for tracing, evaluating, prompt-managing, and deploying agents, LLM applications, and ML models. Open dossier | Self-hosted AI platform and web UI for Ollama, OpenAI-compatible APIs, RAG, Python function tools, model builder workflows, artifacts, web search, vector databases, enterprise auth, observability, plugins, and MCP-adjacent OpenAPI integrations. Open dossier |
|---|---|---|---|---|
| Next stepsDiffers | ||||
| Trust | ||||
| Review status | ReviewedMaintainer reviewed | ReviewedMaintainer reviewed | ReviewedMaintainer reviewed | ReviewedMaintainer reviewed |
| Package trust | Package not verified | Package not verified | Package not verified | Package not verified |
| Source provenance | Source-backed | Source-backed | Source-backed | Source-backed |
| SubmitterDiffers | davion-knight | oktofeesh1 | oktofeesh1 | — |
| Install risk | Review first | Review first | Review first | Review first |
| Notes | Safety ✓ Privacy ✓ | Safety ✓ Privacy ✓ | Safety ✓ Privacy ✓ | Safety ✓ Privacy ✓ |
| Brand | ||||
| Category | tools | tools | tools | tools |
| Source | source-backed | source-backed | source-backed | source-backed |
| Author | Portkey-AI | BerriAI | MLflow Project | Open WebUI |
| Added | 2026-07-09 | 2026-06-03 | 2026-06-03 | 2026-06-18 |
| Platforms | CLI | CLI | CLI | CLI |
| Source repo | — | — | — | — |
| Safety notes | ✓The gateway sits in the path of your LLM traffic and forwards requests to configured providers using the API keys you supply, so scope those provider keys to the minimum needed and store them securely. Self-hosting exposes a local or deployed endpoint; run it on a trusted network or behind authentication, and do not expose an unauthenticated gateway to the public internet. Guardrails, retries, fallbacks, and conditional routing change how and where requests are sent, so review the gateway configuration before using it for production traffic. Caching returns stored responses for matching requests; confirm that cached content is appropriate to reuse before enabling it for sensitive or per-user data. Treat provider responses returned through the gateway as untrusted input for any downstream action, and keep production configuration narrower than local examples. | ✓LiteLLM can proxy requests to multiple model providers, so route and fallback behavior should be reviewed before production use. Gateway deployments can expose model access to teams or applications; configure authentication, budgets, rate limits, and network access intentionally. Avoid logging sensitive prompt, response, or credential material when enabling debugging, observability, or admin features. | ✓MLflow evaluations, traces, judges, and dashboards are review signals, not proof that an agent, LLM application, prompt, model, or deployment is correct, safe, fair, or production-ready. Autologging, decorators, OpenTelemetry ingestion, manual spans, and framework integrations can wrap live application code and record intermediate agent steps, retrievals, tool calls, model requests, and model responses. LLM-as-a-judge scorers and prompt optimization workflows can call configured model providers, consume quota, hit rate limits, and produce evaluator-model errors that require separate handling. AI Gateway and serving workflows can centralize model access, routing, rate limits, and credentials; incorrect configuration can route traffic to the wrong provider or expose more access than intended. Production tracing, async logging, tracking servers, registries, artifact stores, and deployment endpoints should be reviewed for authentication, TLS, network exposure, backups, and incident response before production use. Model registry and deployment workflows can influence real production behavior, so promotion, rollback, and approval rules should be separated from exploratory eval results. | ✓Open WebUI can run Python function-calling tools, RAG ingestion, web search, web browsing, image generation, plugins, and model/provider integrations; review each capability before enabling it for untrusted users. Docker examples expose web ports and persistent volumes. Mount persistent data, set admin/auth controls, and avoid treating demo defaults as production hardening. Python function tools and plugin pipelines can execute application logic and access configured services. Restrict tool creation and plugin installation to trusted administrators. RAG and web browsing can ingest local documents, URLs, cloud files, and extracted text; test indexing quality and permissions before exposing private corpora to users. Open WebUI uses a custom Open WebUI License with branding restrictions and enterprise-license exceptions. Verify license terms before redistribution, white-labeling, or commercial deployment. |
| Privacy notes | ✓Requests routed through the gateway, including prompts and inputs, are forwarded to the configured model providers, which process that data under their own terms. If observability or logging is enabled, prompts, responses, metadata, and metrics can be recorded in the store you configure, so choose a destination and retention policy deliberately. Caching stores request and response data to serve repeated calls; scope and expire the cache appropriately for the sensitivity of the traffic. Provider API keys, gateway configuration, and any request logs should be kept out of version control and access-controlled like other secrets and operational data. | ✓Prompts and responses pass through the LiteLLM process and then to the selected upstream model provider. Gateway logs, spend tracking, and observability integrations may retain request metadata or payload excerpts depending on configuration. Self-hosted deployments still depend on the privacy terms of each configured model provider. | ✓MLflow traces and evaluations can capture prompts, completions, retrieved context, tool arguments, tool outputs, spans, metadata, latency, token usage, costs, scores, datasets, expectations, and human feedback. Agent traces may contain customer data, private documents, source snippets, proprietary prompts, internal identifiers, secrets accidentally passed to tools, or model outputs that need redaction before storage or sharing. LLM-as-a-judge scorers, prompt optimization, AI Gateway calls, and serving endpoints may send prompts, outputs, context, or traces to configured model providers unless a reviewed local or private provider path is used. Tracking servers, backend databases, artifact stores, evaluation datasets, prompt registries, model registries, and exported reports should follow normal access-control, retention, audit-log, and deletion policies. Public demos, notebooks, and examples should not be copied into production workflows with real API keys, raw customer traces, unreleased prompts, or sensitive evaluation data. | ✓Chats, prompts, uploaded files, document chunks, embeddings, vector metadata, web search results, browser-fetched pages, Python tool inputs, plugin outputs, voice/video data, logs, metrics, and traces may contain private data. Configured model providers, vector databases, document extraction engines, web search providers, image providers, object storage, Redis, auth providers, and observability backends may receive user data. Keep provider keys, OAuth/LDAP/SSO secrets, database URLs, object storage keys, plugin credentials, uploaded files, RAG indexes, and OpenTelemetry exports out of public repos and screenshots. Define retention, deletion, tenant separation, group permissions, export policy, and audit review before using Open WebUI as a shared internal workspace. |
| Prerequisites |
|
|
|
|
| Install | — | — | — | |
| Config | — | — | — | — |
| Citations | ||||
| Claim | Unclaimed | Unclaimed | Unclaimed | Unclaimed |
Open-source AI gateway for routing LLM calls across providers.
Trace, evaluate, monitor, and manage agents, LLM apps, prompts, and models.
Source-backed guides for putting this to work.
Instrument LLM and agent apps with traces, metrics, logs, and redaction.
Loading live community signals…
A short, calm digest of reviewed Claude resources. Unsubscribe any time.