Engineering · Claude Code · Security
How we built an LLM API gateway with per-user spend limits
Our LLM API gateway with per-user spend limits gives each member $20 of Claude Opus 5.5 without a key: reserve, settle, per-key budgets and a closed allowlist.
On this page
Every new UseAgent member gets $20 of Claude Opus 5.5 to spend in Claude Code or OpenCode. The money is ours, the Anthropic keys are ours, and the thing spending it is an agent in a sandbox we don't trust. So the piece in the middle is an LLM API gateway with per-user spend limits, and this is how it works, including how it failed on its first day in production.
The sandbox never sees a key
Engines run inside the sandbox next to a shell and whatever packages the agent decides to install. Anything in there can read env vars and files. A provider key put there is a key handed to the model and to every dependency it pulls. We don't do that with member keys and we're not going to start with our own.
Each run gets a provider token instead. It's a short-lived signed capability naming the org, user, thread, run, engine and provider. Claude Code thinks it's talking to Anthropic. It's talking to our gateway, which verifies the token, looks the run up in Postgres, and only then decides which real key serves the call, if any. Outside a live turn the token is inert, whatever its expiry says.
That one hop is what makes the rest possible. The gateway knows who is asking, for which run, with which model, and it writes an audit row per call. Spend limits become bookkeeping on those rows. The sandbox and control plane post has the wider picture.
Who gets the credit
The backend decides at admission and stamps the decision on the run. The gateway reads it and never re-derives it. A run is on the credit when:
- the member has no Anthropic key of their own in that org, in any state (a rejected key never falls through to ours)
- the model is Claude Opus 5.5 on an engine that speaks Anthropic's API
- the member started the task themselves, not an automation, a bot or a delegated child task
- at least $2 is left, about one first call's worst case, so a run we admit isn't refused on its first call
Codex isn't on the credit. It can run on the member's own ChatGPT plan, which we wrote about in running Codex on a ChatGPT plan.
When the $20 runs out, new tasks run on GLM credits from us. That lane has its own keys and its own caps: a per-member monthly cap and a daily cap across everyone on the metered key, and coding-plan keys that rest when Z.ai answers "usage limit reached" until the reset time it states. Z.ai states that time in China Standard Time. The parser converts it.
An LLM API gateway with per-user spend limits has to reserve first
Our first spend ledger, in September, charged once per run at settlement and refused new runs once a member hit their allowance. It never stopped anything mid-turn. That took eleven review rounds, and rounds seven to ten found fourteen new money bugs, all in two mechanisms: accounting for turns stopped halfway, and Chat charging. We cut both into a follow-up rather than ship a fifteenth fix. Charging after the fact is fine when the member pays the provider directly. It doesn't work when the key is ours, because a few parallel streaming calls can spend far past a balance before the first one reports usage.
So every credit call reserves its worst case before it leaves, and settles at the real price after.
input = (request body bytes - base64 image and PDF bytes) / 2
+ 5,811 tokens Anthropic adds (tool-use system prompt, client tool definitions)
+ 4,784 per image (the high-resolution ceiling)
reserve = (input × dearest input rate + max_tokens × output rate) × 1.1Image and PDF data is taken out first, since it is already priced per image or per page, and counting its base64 as text once refused members who still had half their credit. Bytes, not characters, because CJK text and emoji run three to four bytes a character. The dearest input rate on Opus 5.5 is the one-hour cache write at $8 per million tokens. Output is $20 per million. The 1.1 is Anthropic's US-only inference multiplier, applied whether or not the call ends up there. A call with 100K tokens of input asking for 64,000 output tokens reserves about $2.29. A real first Claude Code turn settles at about $0.35, most of it writing a 69K-token prompt cache.
PDFs get special handling. A PDF's page count comes from the file, and we don't price off something the file claims about itself. When a request carries one, the gateway asks Anthropic's count_tokens endpoint first. If that fails, it reserves the most pages Anthropic takes in one request, 600.
Reservations live in the gateway audit table and happen under a Postgres advisory transaction lock. One lock for every credit admission across every member. That sounds bad and isn't: the key budgets span all members, and the volume is a few hundred dollars of calls. The code has a comment saying exactly that, with the upgrade path if it ever matters.
A reservation comes back with one verdict:
| Verdict | Meaning |
|---|---|
| fits | reserved on the key with the most budget left |
| busy | it would fit once the member's calls in flight settle |
| member_short | the member has credit left, less than this call needs |
| member_cap | the member's $20 is spent |
| total_cap | no key has room |
Settlement starts from what Anthropic reported. An error costs nothing, since Anthropic bills none. A finished message is priced from its usage block, cache reads, cache writes and web searches included. A stream that never finished, a member pressing stop for example, pays the input Anthropic reported plus every output token it asked for, capped at its reservation. With no usage reported at all, it pays the reservation. A call that never left the box (connection refused, DNS failure) releases its reservation. And a call still marked started past the gateway's upstream timeout counts at its reservation, because the gateway that sent it died and nothing will ever settle it.
The SSE reader that pulls usage out of the stream is incremental and bounded. An event past the size bound gets dropped and taints the read. A tainted stream never counts as finished, so it pays its reservation.
Budgets per key and a cap on calls in flight
The estimate is still an estimate. Anthropic can bill more input than our ceilings in edge cases. Two things bound that error.
Each house key has a budget set a little under its real balance. Every member's credit together stops at the sum of those budgets, and the headroom underneath absorbs estimate error. When Anthropic answers 401, 402, 403 or "credit balance is too low" for a key, the gateway parks that key for an hour, releases the reservation (the refused call cost nothing) and sends the call once more on the next key.
A member also gets at most three credit calls in flight. That number is the real bound on how far a cap can overshoot: three calls' worth of estimate error, at most. A fourth call waits as busy.
A closed allowlist of request shapes
This part took the most review. You can only price a request you understand, so a credit call may only send shapes on a closed list:
- known top-level fields (model, messages, system, tools, thinking, max_tokens and a dozen more)
- known content blocks: text, image, document, tool use, tool result, thinking
- custom tools, Anthropic's client tools (bash, text editor, computer), and web search with a bound
- at most 64 levels of nesting
Anything not on a list is refused before a reservation exists. A new Anthropic feature never bills us unseen.
Betas go the other way. Every engine release adds a few, and refusing unknown ones would mean refusing the next Claude Code release. An unknown beta is dropped from the forwarded header and logged once per run. Without it, Anthropic either serves the request at the normal price or refuses it unbilled. A beta that changes the price, fast mode for example, never goes on the list.
The gateway forwards exactly the bytes it checked. It parses the engine's body, validates it, re-serializes it and sends that, never the engine's original bytes. There's no gap between what we priced and what Anthropic reads.
Web search has its own math. It's a server tool: Anthropic runs a loop on its side for up to ten sampling rounds, and every round reads the input, every result so far and the output so far again. Results have no documented size ceiling. So web search is allowed only with max_uses between 1 and 10, only on calls with at most 20K tokens of input (Claude Code's WebSearch sub-call is one short message, about 6,000 tokens once Anthropic adds its part), and it's reserved as if all ten rounds run and each search returns 20,000 tokens, plus $0.01 a search. That 20,000 is our number, not Anthropic's. If settled searches report more, it goes up.
About 290 tests passed and the first real run failed
The design above went through about ten adversarial review rounds and around 290 tests. On October 13 we admitted real production runs onto the credit and both engines failed.
Claude Code printed "Claude API authentication failed" ten times, then gave up with no progress. OpenCode failed with output_limit_exceeded. The gateway audit table had zero rows for them.
Zero rows was the useful clue. No call had reached the credit code at all. Both engines were asking Opus 5.5 for max_tokens 128,000, and the gateway's output ceiling was 65,536, so the request policy refused with a 403 before an audit row existed. Claude Code reads a 403 from what it believes is Anthropic as a broken credential and says so.
Every one of those 290 tests used a simulated transport. They sent requests we wrote, shaped like what we believed the engines send. None of them ran a real engine binary in a real sandbox against the real gateway, and a real engine sends its own default max_tokens.
A second mismatch sat behind the first. We'd captured the beta allowlist from the engine versions pinned in our sandbox image. Production sandboxes install engines at runtime, and they were running Claude Code 2.1.296 and OpenCode 2.0.26, not the pins.
The fix was small:
- one shared engine output limit of 64,000 tokens, set through Claude Code's managed settings and OpenCode's model limits
- unknown betas dropped instead of refused
- an over-limit request gets an Anthropic-shaped 400, which the engine reports for what it is
The credit went live at 15:47 UTC the same day, after one real Claude Code run and one real OpenCode run on production answered, and their audit charges matched list prices exactly.
The rule we kept: no feature that spends money gets its keys in production until each engine has done one real run against the real gateway. Simulated transports prove your logic. They tell you nothing about what a binary you didn't write actually sends.
What's still rough
The reservation isn't a strict bound. A member's cap can overshoot by the estimate error of three calls in flight, and the per-key headroom is what pays for that. One advisory lock serializes every credit reservation, which is fine at this volume and the first thing to split if it grows. The 20,000-token guess per search result is a guess.
The $20 is waiting at sign-up. Pick Claude Opus 5.5 in Claude Code or OpenCode and give it something real to do.
Frequently asked questions
How do you enforce per-user spend limits on an LLM API?
Put a gateway between the agent and the provider, give each run a signed token instead of a key, and reserve each call's worst-case cost before forwarding it. Settle at the usage the provider reports, and serialize reservations in the database so parallel calls cannot pass a cap.
Why does Claude Code say "Claude API authentication failed" behind a proxy?
Claude Code reports a 403 from its API endpoint as an authentication failure. In our case the proxy refused a max_tokens of 128,000 above its 65,536 ceiling. Check your proxy's output-token limit and return an Anthropic-shaped 400 for over-limit requests.
Which UseAgent engines can use the $20 Claude Opus 5.5 credit?
Claude Code and OpenCode, on tasks you start yourself, if you have no Anthropic key of your own connected. Codex can run on your own ChatGPT plan. When the credit is used up, new tasks run on GLM credits from us.