Skip to content
UseAgent

Engineering · Testing · Reliability

How we load test AI coding agents: 50 runs at once

How we load test AI coding agents: 50 concurrent OpenCode runs on E2B, the fleet caps that had to move, what failed, and the soak storms behind them.

Abhishek7 min read
On this page

On 2026-09-10 we wanted to know if production could run 50 agent sandboxes at once. The task was deliberately boring, "find a stock price" on OpenCode, because we wanted to load test AI coding agents on our infrastructure, not on a hard prompt. The short answer: the platform held, the model's rate limit didn't, and four sandboxes never got created.

This is how we ran it, what the numbers were, and the soak harness that made us trust the result.

How we load test AI coding agents through the public API

Everything went through the public API, the same way a customer's script would. We minted an org API key, sent POST /api/runs with a bearer token and {prompt, engine, model}, and polled GET /api/runs/:id/timings, which returns per-stage spans and marks with durations. Every prompt started with [TEST] so it was easy to find and clean up afterwards. Creating 50 runs took about three seconds.

The sandboxes ran on E2B, where our plan allows 100 concurrent.

Batch 1: our own caps were the first wall

The first batch used one model, a Qwen 3.8 27B route on Cerebras, and production's caps as they were.

Ten of the 50 creates came back 429 fleet_queue_full. The per-org queue depth defaulted to 40. The host budget then capped concurrency at 13 sandboxes: we size admission by declared CPU and memory per sandbox, and the configured host CPU budget only had room for 13. So 40 runs queued behind 13 slots, and the median queue wait was 243 seconds.

24 runs completed. 9 failed, every one with "Tokens per minute limit exceeded" from the model provider. With that much contention the median model turn took 195 seconds.

One thing worked as designed. When a run finishes, its sandbox stays retained for the thread's next turn, and under queue pressure the fleet reconciler reclaimed those retained sandboxes about 30 seconds after each completion, so the queue kept moving.

Before batch 2 we raised the global, per-org and dispatch caps to 50, the queue depth to 100, and the host CPU and memory budgets far enough that they stopped being the limit.

Batch 2: 50 runs, three models

Metric Result
Runs created 50 in 3 s
Running within 7 s 38
All admitted within 70 s
Completed 45
Failed 5 (4 sandbox create timeouts, 1 token limit)
Time to done p50 90 s, p90 254 s, max 274 s
E2B's count of running sandboxes 51

Model turn medians split cleanly by model: Gemini 3.7 Flash 18 seconds, a preview model 69 seconds, Qwen 3.8 27B 133 seconds. The model decides most of the spread between runs.

The four create failures were [deadline_exceeded] during "Provisioning cloud sandbox". E2B's create or connect timed out under a burst of 48 creates, and the control plane did not retry a failed create. That one is on us, and the fix is a single retry.

Where the time goes on a cold sandbox

The timings ledger showed the harness cost per run on a cold sandbox:

text
sandbox create          1.7 s
provider bootstrap     31   s   (22 s in batch 1)
first shell            12   s
session start           1.8 s
-------------------------------
before the model is asked  ~45 s

Provider bootstrap means getting the engine runtime ready inside a fresh sandbox. It took 22 seconds with 13 sandboxes starting and 31 with 50. It is the biggest piece of overhead under load and the one we're cutting, mostly by baking more into the image. E2B vs Daytona has the startup work in detail.

The control plane barely noticed

During batch 2 the backend used about 25% of one core and 1.4 GB of memory. The gateway was idle. Host load was 1.1 on 12 cores.

That's the expected shape when the control plane only orchestrates. Agents run in their own sandboxes, models run at the provider, and the backend writes events and moves state. So the order of fixes after this test was: retry a timed-out sandbox create once, keep a warm pool of two or three sandboxes instead of one, and cut cold provider bootstrap.

A month later, during a proxy upgrade, we ran 40 and then 50 concurrent runs again to load the health checks. The lightweight liveness endpoint stayed at a p99 of 348 ms at 40 runs and 99 ms at 50. The full health endpoint, which pings the database, peaked at 1.9 seconds, which is why the proxy checks liveness and not full health. Same test, another finding: retained sandboxes still counted against the fleet cap for about 10 minutes after their run, so the second wave queued for about 9 minutes.

Two earlier capacity failures

A shared memory cap. In August, before production ran on E2B, a 50-run evaluation on a shared Daytona organization pushed the pool from 34 to 76 sandboxes and hit Total memory limit exceeded. Maximum allowed: 500GiB. That cap is a total across the whole org. About 24 of our 50 runs never got a sandbox and failed in roughly four seconds, and runs from other sessions sharing the org failed with them. Now we list what's already running before any fan-out and size to the headroom.

A budget that counted sleeping sandboxes. A single new thread once waited 33 seconds for admission. The admission check counted every thread's retained sandbox against the host budget for its full 72-hour retention, though the provider had paused it after an hour, and the old budget had room for five 2-vCPU sandboxes. We raised the host budget and gave retained sandboxes their own, much shorter reservation window.

The same budget logic broke our CI. Backend tests ran with the production host budget. Tests that simulate crashed workers leave sandbox leases active at 2,000 millicores each, the reconciler is off in tests, and after six of them every later run in the suite queued on capacity until some other test happened to truncate the lease table. Bun's file order decided which tests landed in that dead window, so a different set failed each run and nothing reproduced locally. We first raised test timeouts, which only hid it. The fix: the test preload opens the host budget.

Before load, we soaked

Real-model load tests are expensive and noisy. Before trusting one, we ran a soak harness against the core: five storm scripts plus a sixth for memory capture, each in its own subprocess with a fresh throwaway database, rotated by an orchestrator that folds results into one report. One 153-minute session on 2026-08-06 ran 202 rotations:

Storm What it does Volume Failures
conversation Kicks plus threaded replies, sequential and concurrent; checks strict per-thread order, one turn in flight, no drops or duplicates 509 threads, 2,545 runs 0
idempotency N concurrent POSTs with the same key; expects exactly one run and clean replays 5,304 POSTs 0
crash SIGKILL at random phases: early, mid-run, pre-settle, mid-outbox 135 kills 1
outbox 5,100 Slack and 5,100 memory deliveries with injected 429s, 500s and permanent errors 4,658 delivered, 442 dead-lettered per lane 0
native High-frequency event capture with random reconnect-cursor replays; expects zero missing sequence numbers 272 runs, 9,002 frames 0

The crash storm also recorded one duplicate Slack delivery. Our reply outbox is at-least-once by design, so a kill at the wrong moment can repeat a message. We'd rather repeat one than lose one. The memory-capture storm's failures in that report were the harness's own timing, killing the backend before the request was in flight, and the at-most-once checks themselves passed.

The best find came on the first day, from the idempotency storm, not the crash one. Twenty concurrent POST /api/runs with the same Idempotency-Key returned 500,201,500,500,...: one winner and a pile of 500s. Data was fine, exactly one run was created, but the contract was broken.

The cause was a wrapped error. Drizzle throws a DrizzleQueryError, whose code is undefined; the Postgres SQLSTATE 23505 lives on err.cause. Our isUniqueViolation read only the top-level code, so the losers of the insert race skipped the replay path and threw. The fix walks the cause chain up to five levels, and because the predicate is shared it fixed every caller at once. Re-run at 40 concurrent across three rounds: zero failures. The storm stays in rotation.

What it found is the argument for soaking. Unit tests hit that code path one request at a time, where the fast path catches duplicates before the insert. Only real concurrency gets a loser past the fast path and into the race.

The soak harness lives in the open-source repo under backend/test/e2e/soak; run it against a throwaway database before you point real models at anything.

Frequently asked questions

How many concurrent AI coding agent runs can one control plane handle?

In our test, 50 concurrent OpenCode runs cost the backend about a quarter of one CPU core and 1.4 GB of memory, with host load at 1.1 on 12 cores. The limits we hit were the model provider's per-minute token budget and sandbox create timeouts.

Why do concurrent requests with the same idempotency key return 500?

In our case Drizzle wrapped the Postgres unique-violation error, so the SQLSTATE 23505 sat on err.cause and our check only read err.code. Walking the cause chain fixed every caller at once.

How long does a coding agent sandbox take to start under load?

In our 50-run test on E2B, create took 1.7 seconds, provider bootstrap 31 seconds, the first shell 12 seconds and session start 1.8 seconds, so about 45 seconds before the model got the prompt.

All posts →

Hand your next task
to a coding agent.

UseAgent runs Claude Code, Codex and OpenCode in cloud sandboxes. The core is open source, so you can read the code or host it yourself.