Spot GPU Instances: Big Savings, and When They Backfire
The Deal: Same GPU, Fraction of the Price
A spot instance is spare data center capacity sold at a steep discount — and GPUs are where the discounts get dramatic. The same card that costs dollars per hour on-demand can often be rented at 60% to 90% off, and the hardware is identical. The catch is written into the deal: the cloud can take it back at any time, usually with only about two minutes of warning.
For anyone who has watched GPU bills eat a project budget alive, that discount is hard to ignore. But the price tag is only half the story — whether spot works for you depends entirely on what you are running.
Why Clouds Sell GPUs So Cheap
The economics behind spot is not charity. A data center full of GPUs that sits idle earns nothing, and idle capacity is a cloud’s biggest sunk cost. Spot pricing turns that liability into whatever the market will pay — better to fill a machine at 20% of list price than let it warm an empty rack.
This also explains the volatility. When on-demand customers come back, your bargain instance is the first to go, because full-price customers always win. Spot capacity is not a product the cloud promises to keep; it is leftovers they monetize until needed. Recent AI demand has made this more visible than ever — as we covered in “Why AI Workloads Are Rewriting Data Center Power Budgets”, GPU capacity is now the scarcest resource in the building, and scarcity is exactly what makes reclaimed spot GPUs common.
Where Spot Shines
Some workloads barely notice the eviction notices:
Batch jobs and data processing. Walkthroughs over datasets, ETL pipelines, media encoding — if the job can stop and resume, losing a machine mid-run costs you minutes, not days.
Fault-tolerant training. Deep learning frameworks have mature checkpointing: save the model state every few minutes to object storage, and an interrupted run restarts from the last checkpoint. Many large-scale training pipelines are explicitly designed this down-top, running cheap on spot by default.
CI and test grids. Running test suites at 3 AM on spot GPUs costs a fraction of office-hours prices, and a flaky test machine is an annoyance, not a disaster.
Overflow capacity for inference. Off-peak inference traffic can ride spot instances behind an autoscaler; when capacity vanishes, traffic falls back to on-demand nodes.
Experiments and side projects. Research sketches where “the result tomorrow” beats “the result tonight” — perfect spot territory.
When It Backfires
The failure modes are predictable, and they all sting:
Long uncheckpointed training dies. A multi-day training run without recent checkpoints loses hours of progress at every eviction. Three evictions in a week can turn a 5-day project into a 10-day one — quietly erasing the savings.
Stateful services fall over. Databases, queues, or long-lived in-memory caches on spot instances turn every eviction into an incident. If the service holds state, spot is the wrong tier, full stop.
Deadline-driven work gets gambled away. A demo, a launch, or a paper deadline means the schedule, not the bill, is what you cannot afford to break. Spot introduces a lottery element precisely where you need certainty most.
Silent capacity droughts. Sometimes spot capacity is simply unavailable in your region — the price looks fine, the launch just never succeeds. Teams discover this during a crunch, which is the worst possible time.
Playing It Safe: Checkpoints and Diversity
If spot fits your workload, a few habits keep it profitable:
Checkpoint religiously. Every few minutes, to durable storage. This is the single habit that converts spot from a gamble into a bargain.
Diversify. Spread work across instance types, sizes, and availability zones. A fleet of mixed spot capacity survives regional droughts far better than fifty copies of the same SKU.
Use an orchestrator. Managed spot orchestration handles bidding, replacement, and diversification automatically — most clouds offer it, and it removes most of the manual pain.
Always have a fallback. On-demand capacity as the safety net, spot as the discount layer. The architecture should assume evictions happen and treat them as routine.
How to Decide: Spot, Reserved, or On-Demand
The three pricing tiers map cleanly onto three situations:
Spot for interruptible, checkpointed, bursty work — the discount is real and the risk is managed.
Reserved or committed-use for steady, predictable 24/7 load: you trade flexibility for a guaranteed discount, and you never worry about evictions.
On-demand for everything in between — steady enough to hate interruptions, short enough that commitments do not pay off.
And there is a fourth path worth knowing: fixed-configuration GPU rentals from specialized providers. Renting a known card on a known machine for a fixed term trades the spot lottery and the on-demand premium for predictability — for example, VMRack’s GPU rental service offers everything from the mature H100 to the latest-generation B300 Blackwell, single card or full cluster, which suits teams that want to stop thinking about pricing games and just hold hardware. For teams that cannot tolerate any eviction but also cannot justify reservation math, that middle ground often lands exactly right.
The Bottom Line
Spot GPU instances are one of the best deals in cloud computing — for the workloads shaped like them. Checkpoint your runs, keep state off spot, diversify your fleet, and the 60-90% discount is yours. Ignore those rules, and the money you saved on compute you will pay back in delays, incidents, and deadline roulette.
Thank you for reading all the way to the end!
