LiteLLM comes up in almost every conversation about self-hosted LLM gateways, and it should — it's the library a large share of this category, OpenProxyAI included, is built on top of for provider abstraction. So the honest framing isn't "OpenProxyAI vs. LiteLLM" as competitors. It's closer to: LiteLLM is the engine, and OpenProxyAI is what you get when you build the rest of the car around it.

Portkey is the most feature-comparable product to OpenProxyAI in this space — a unified gateway, real guardrails (including PII redaction), observability, and enterprise governance. If you put the feature lists side by side, they overlap a lot. The difference that actually matters is architectural, not a checklist item: which deployment model is the default, and which one you have to pay to unlock.

We shipped three compliance templates this phase — Healthcare, Finance, and Government — and the most important thing to say about them is what they're not: they're not a claim that OpenProxyAI holds any certification on your behalf. They're pre-built policy configurations you apply, review, and enforce yourself.

"Adds a few milliseconds of overhead" is the kind of line that shows up in every proxy's marketing copy, ours included. It's true, but it undersells what's actually happening in that window. Six things happen, in a specific order, and the order is load-bearing — get it wrong and you either leak data you meant to redact, or you pay for cache infrastructure you can't actually use.

Every multi-tenant system has to answer the same question: what stops a bug in organization A's code path from returning organization B's data? The conventional answer is discipline — every query gets a WHERE org_id = ? clause, added by convention, reviewed in code review, hopefully never forgotten.

We didn't want the answer to be discipline. We wanted it to be a database property that doesn't care whether the application code remembered.

Most AI gateways you'll evaluate are built SaaS-first: a multi-tenant service you send your traffic to, with a self-hosted or "VPC" option available if you pay for the enterprise tier. OpenProxyAI is built the other way around. Self-hosted is the default, not the upsell.

That ordering isn't a licensing decision. It changes what you're actually agreeing to when you adopt the tool.