At 2:14 a.m. on a Tuesday, traffic to one of our customers' image-generation apps jumped 60x in eleven minutes. By sunrise they were serving more requests per hour than they had in the entire previous month. Nobody on their team was paged.

Elastic by default

Codexa scales GPU workers based on queue depth and latency targets, not fixed instance counts. As requests piled up, the platform added capacity in steps of 50 GPUs, peaking at just over 1,000 within nine minutes. When traffic eased, it scaled back down to zero.

What made it work

• Model weights were cached close to every GPU pool, so new workers loaded in seconds.

• Requests were routed to the nearest region with free capacity.

• Per-project quotas stopped the spike from affecting other teams' workloads.

What we learned

Scaling up is the easy part. Scaling up without surprising anyone on the bill is harder. We've since added spend alerts that fire before a budget is reached, and a dashboard that shows exactly which endpoint is driving cost.

We went from 300 to 18,000 users overnight and our infrastructure bill made sense the next morning. That's the dream. — Arjun Patel, founder of Lumen Studio

If you're preparing for a launch, our team can review your scaling settings before the big day. Just reach out.