Modeling the Power Bill Behind Your GPU Cluster

Power usage effectiveness is the ratio of total facility power to the power the compute itself uses; say a PUE of 1.4 means your cooling, lighting, and facility overhead adds 40 percent on top of what the GPUs draw. A colocation facility will usually quote you this number, and it's worth asking for it explicitly rather than assuming it, since the difference between a well-run facility and an average one can be a meaningful share of your total power cost.

Say a GPU draws 700 watts under load and you're running 500 of them continuously, that's 350 kilowatts of compute power; at a PUE of 1.4 your facility draws 490 kilowatts total, and if your utility rate is 10 cents per kilowatt-hour, that's roughly $1,176 an hour, or a bit over $850,000 a month, before you've counted a single dollar of the GPU lease itself.

To turn GPU draw into a monthly power bill, work through these steps:

1. Take each GPU's draw under load and multiply by the number of GPUs running continuously to get total compute power.
2. Multiply by the facility's PUE, which you should ask the colocation provider to state explicitly, to get total facility power.
3. Multiply by your utility rate per kilowatt-hour and by the hours in the billing period to get the energy charge.
4. Add demand charges based on your peak draw, since they can dominate the bill for spiky training workloads.
5. Replace nameplate assumptions with real utilization from your scheduler and rebuild the model as that utilization changes.

Demand charges, a separate fee based on your peak power draw in a billing period rather than total energy used, can dominate the bill for a workload that spikes rather than running flat, which most training workloads do. Negotiating a flatter draw profile, or a utility contract structured around your actual usage pattern rather than a generic industrial rate, is often worth more than shopping for a marginally cheaper per-kilowatt-hour rate.

Some utilities also offer interruptible or curtailable-load rates, a lower price in exchange for agreeing to reduce draw during grid stress events, which can fit a training workload that can tolerate being paused far better than it fits a workload serving live customer traffic. Whether that tradeoff makes sense depends entirely on how tolerant your specific workload is to an unscheduled pause, which is a technical question worth asking before you assume the discount is free money.

A model that assumes every GPU runs at full draw around the clock overstates the bill for a cluster that actually sits idle between training runs, and understates it for one running dense, back-to-back jobs. Pull your real utilization rate from your scheduler, not an assumed number, and apply it to the power model, since the gap between nameplate power draw and actual metered draw is often the single largest source of error in a first-pass estimate.

For example, imagine two clusters of identical size. One runs dense, back-to-back training jobs, while the other sits idle between occasional runs. A model built on nameplate draw gives both the same bill, which overstates the second and understates the first. A practical decision rule: build the model from metered draw for at least one full billing period, split the result into energy charges and demand charges, and flag any month where the demand portion looks unusually large. That flag tells you whether your cost problem is how much power you use or when you use it, and those two problems have different fixes.

A colocation or utility agreement with demand-charge structure, PUE commitments, and any power-cost pass-through clauses needs to be reviewed and signed with the same care as any other multi-year commitment this size, not rushed because it's an infrastructure decision rather than a real estate one. Foxit eSign is a reasonable place to manage the signature and amendment trail on agreements like this; Process Street can hold the actual due-diligence checklist your team runs before signing one, so the review doesn't depend on whoever did it last time remembering every clause to check.

A power and cooling model built at launch, when utilization is low, understates the real cost once the cluster is running closer to capacity, since fixed facility overhead gets spread across more actual compute hours. Rebuild the model against actual metered draw at least twice in the first year, not just once at the planning stage, since the assumptions that looked reasonable before launch rarely survive contact with real utilization.