GPU cloud pricing explained: what you are actually paying for

The headline rate is a fraction of the bill

Every GPU cloud advertises a number in dollars per hour. It is the easiest thing to compare and the least useful, because the compute rate typically accounts for somewhere between half and two thirds of what a working AI team actually spends.

The rest is storage, egress, idle capacity, retries, and time. Time is the largest and least visible of these — an engineer waiting three weeks for capacity costs more than the capacity.

None of this means providers are being dishonest. It means the unit being advertised is not the unit you consume.

The line items that surprise people

In rough order of how often they cause a difficult conversation with finance:

• Idle instances. The single largest source of waste in every GPU environment we have seen. A machine left running over a weekend costs the same as one doing useful work. Not a pricing problem — a controls problem.
• Storage that outlives the compute. Persistent volumes keep billing after the instance stops, which is precisely the behaviour you want and precisely what nobody budgets for. A stopped project with two terabytes attached still costs money every month.
• Egress. Moving data out of a region is usually billed per gigabyte. Teams that keep their datasets with one provider and their compute with another discover this in month two.
• Failed and repeated runs. A training run that crashes at hour nine is billed for nine hours. Runs that did not checkpoint get repeated from zero.
• Inference retries. If a model returns malformed output a fifth of the time and your client retries, you are paying for 1.2 requests per request, forever.
• Overprovisioned GPU class. Running a workload on the largest available accelerator because it was available, when a smaller one would have finished in a comparable time for materially less.

How each service is actually metered

Different parts of an AI workload bill on different units, which is why a single hourly rate cannot describe the cost.

• Compute — per second or per hour of instance runtime, whether or not the GPU is busy
• Block storage — per gigabyte-month, billed while the volume exists
• Object storage — per gigabyte-month plus request volume
• Inference — per input and output token, with output usually costing more than input
• Document ingestion — per page, with OCR sometimes billed separately
• Egress — per gigabyte leaving the region

The practical consequence is that a training-heavy team and an inference-heavy team have almost nothing in common in their cost structure, and advice written for one is misleading for the other.

Reserved versus on-demand

On-demand capacity costs more per hour and commits you to nothing. Reserved capacity costs less per hour and holds hardware for a window whether or not you use it.

The decision is about utilisation, not about price. If your GPUs are busy more than roughly 60 to 70 percent of the time, reserved capacity is straightforwardly cheaper. Below that, you are pre-paying for idle hardware and on-demand wins.

The failure mode is reserving capacity for a project that has not yet proven it will run continuously. Start on demand, measure your actual utilisation for a few weeks, then reserve against a number rather than against an intention.

Why some providers do not publish rates

It is worth being straightforward about this, since NXAARA is one of them.

GPU rates move with hardware availability, with what capacity a provider has secured, and with what they have committed to existing customers. A published rate that is honoured only when convenient is worse than no published rate, because it converts a comparison into a negotiation after the person has already invested time.

The reasonable standard is that you should always see a firm rate before you launch anything, and it should be the rate you are billed. Where that appears — a public page, a console, a written quote — matters less than whether it holds. When you are comparing providers, ask what happens to your rate if their capacity tightens next quarter. The answer is more informative than the number.

Modelling your own cost before committing

A rough model built in an hour beats a precise one built after the invoice. Estimate these five things.

• Training: hours per run × runs per month × instance rate. Add 20 percent for failed and repeated runs unless you checkpoint rigorously.
• Storage: dataset size + checkpoint size × number of retained versions, per gigabyte-month. Checkpoints are usually the larger number and the one people forget.
• Inference: requests per day × average input tokens + output tokens, at the per-token rate. Multiply by your retry rate.
• Idle: honestly estimate the hours per week a GPU will be running without work. If you have no budget controls, assume it is high.
• Egress: how much data leaves the region per month, and why.

Then ask which of these you can reduce with controls rather than with a discount. Spend ceilings, idle alerts and per-key attribution routinely cut a bill more than a negotiated rate does, and you can implement them this week.