How the system is put together and why.
View All TagsLiteLLM comes up in almost every conversation about self-hosted LLM gateways, and it should — it's the library a large share of this category, OpenProxyAI included, is built on top of for provider abstraction. So the honest framing isn't "OpenProxyAI vs. LiteLLM" as competitors. It's closer to: LiteLLM is the engine, and OpenProxyAI is what you get when you build the rest of the car around it.
"Adds a few milliseconds of overhead" is the kind of line that shows up in every proxy's marketing copy, ours included. It's true, but it undersells what's actually happening in that window. Six things happen, in a specific order, and the order is load-bearing — get it wrong and you either leak data you meant to redact, or you pay for cache infrastructure you can't actually use.
Most AI gateways you'll evaluate are built SaaS-first: a multi-tenant service you send your traffic to, with a self-hosted or "VPC" option available if you pay for the enterprise tier. OpenProxyAI is built the other way around. Self-hosted is the default, not the upsell.
That ordering isn't a licensing decision. It changes what you're actually agreeing to when you adopt the tool.