Skip to main content

2 posts tagged with "Engineering"

Implementation notes and technical deep-dives.

View All Tags

What actually happens in the four milliseconds before your LLM call

· 3 min read
Founder, OpenProxyAI

"Adds a few milliseconds of overhead" is the kind of line that shows up in every proxy's marketing copy, ours included. It's true, but it undersells what's actually happening in that window. Six things happen, in a specific order, and the order is load-bearing — get it wrong and you either leak data you meant to redact, or you pay for cache infrastructure you can't actually use.

Row-level security: making data leaks impossible by construction

· 3 min read
Founder, OpenProxyAI

Every multi-tenant system has to answer the same question: what stops a bug in organization A's code path from returning organization B's data? The conventional answer is discipline — every query gets a WHERE org_id = ? clause, added by convention, reviewed in code review, hopefully never forgotten.

We didn't want the answer to be discipline. We wanted it to be a database property that doesn't care whether the application code remembered.