AI Rack Density Is Not a Facility Power Plan

The number that now dominates AI-infrastructure conversations is rack density. A 40 kW rack sounds demanding. A 120 kW rack sounds transformative. A future 200 kW rack sounds like a new asset class.

All three statements can be true, but none is a facility power plan.

My view is that rack density should be treated as the first input to an infrastructure model, not as a shortcut for site capacity. The correct question is not “Can this building host a 120 kW rack?” It is “Can the whole electrical, cooling and operating chain support the intended cluster at the required load, redundancy and delivery date?”

That distinction matters because a rack specification sits inside several larger boundaries: the compute domain, the IT hall, the cooling system, the critical-power path, the building and finally the utility connection. A decision made at one boundary can fail at the next.

Start with the vendor envelope, not a generic density assumption

NVIDIA’s current Mission Control documentation gives an approximate full-load total of 120 kW for a GB200 NVL72 rack, including nodes and rack components. Its DGX GB hardware guide describes the physical system behind that number: 18 compute trays, nine NVLink switch trays, management switches, power shelves, a bus bar and liquid-cooling manifolds in an NVL72 rack.

Those two sources already tell us why “GPU power” and “rack power” are not interchangeable. The rack contains far more than accelerators. Network fabrics, Grace CPUs, power conversion and supporting components all contribute to the electrical and thermal envelope.

The operating model matters too. NVIDIA’s Dynamic Power Software pilot represents GB200 racks with 125 kW rack PDUs and a 375 kW power domain for a three-rack envelope. That is a control example, not a universal facility design, but it shows that power has to be modelled at multiple levels rather than reduced to a single nameplate figure.

Before using any density assumption, I would therefore lock four definitions:

1. the exact rack and compute configuration; 2. whether the stated load is maximum, design, capped or measured; 3. which network and rack components are included; 4. the operating profile expected from the workload.

Without those definitions, a calculation can look precise while mixing incompatible scopes.

A reproducible calculation: from 48 racks to the meter

Consider a hypothetical first phase with 48 GB200 NVL72 racks. This is an illustration, not a claim about a live PowerlandMap project.

Using NVIDIA’s approximate 120 kW full-load rack total:

48 racks × 120 kW = 5,760 kW, or 5.76 MW of rack-level IT load.

That is not yet the full IT load. Suppose the design team separately estimates 0.60 MW for cluster networking outside the rack boundary, storage, control systems and other IT equipment. This 0.60 MW is an explicit project assumption, not an NVIDIA figure.

5.76 MW + 0.60 MW = 6.36 MW of modelled IT load.

Now apply an illustrative design PUE of 1.20:

6.36 MW × 1.20 = 7.632 MW at the facility meter.

The 1.272 MW difference is the modelled non-IT facility load at that operating point. It covers the aggregate overhead represented by PUE, not a detailed equipment schedule. It must not be allocated mechanically to cooling, UPS losses, lighting or pumps without a separate engineering model.

This calculation is useful because every assumption is visible. Change the workload cap, rack count, ancillary IT load or PUE and the answer changes. A buyer can reproduce it in a spreadsheet and compare it with the electrical single-line diagram, cooling design and utility offer.

It is also deliberately incomplete. It does not include construction margin, future halls, black-start strategy, maintenance states, battery charging, generator auxiliaries or the difference between normal and degraded operation. Those belong in the next layer of design.

The wider lesson is consistent with Data Center Capacity Is Not One Number: rack load, IT capacity, facility demand and utility capacity answer different questions.

Utility MW sets a ceiling, not an IT entitlement

The same calculation can be run backwards. If a site has a verified 10 MW maximum import capacity and the design case uses a PUE of 1.20, the arithmetic ceiling is:

10 MW ÷ 1.20 = 8.33 MW of IT load.

That is a theoretical conversion, not a promise of sellable IT capacity. It assumes the full 10 MW can be used continuously for the data center and ignores operating reserves, auxiliary conditions, phasing, transformer limits and contractual restrictions.

This is why a power deal is not IT capacity. A connection agreement, transformer rating and energised operating envelope can all carry the same “MW” label while describing different states.

For AI infrastructure, the gap can become commercially material very quickly. At 120 kW per rack, a 1 MW error is equivalent to more than eight rack envelopes. If the commercial plan assumes those racks are available, the error affects compute revenue, delivery commitments and the value assigned to the powered site.

The cooling boundary must match the electrical boundary

High-density racks make liquid cooling central to the calculation, but “liquid cooled” is still not a complete specification.

The design needs temperatures, flow, pressure, water quality, redundancy and the split between liquid and residual air cooling. The rack manifold is an interface. The facility must still reject the heat through coolant distribution units, pumps, heat exchangers, chillers or dry coolers, depending on climate and design.

Schneider Electric’s AI factory reference-design catalogue includes a 7.392 MW GB200 NVL72 design with chilled water and liquid-cooled AI clusters. Schneider also describes densities around 132 kW per rack in its overview of NVIDIA-aligned designs.

These are useful engineering references, but they should not be copied into an unrelated site. A reference design is a coherent package of assumptions. Climate, redundancy, water strategy, voltage, equipment availability and local code can change the result.

The practical diligence question is whether the rack, CDU, secondary loop and heat-rejection plant have been sized to the same load case. Our earlier analysis, Cooling Begins at the Interface, explains why a missing boundary condition can be more dangerous than a missing equipment brand.

Water also requires scope discipline. Direct-to-chip cooling does not automatically mean high site water consumption, and it does not automatically mean zero water consumption. The answer depends on the heat-rejection architecture and climate. Liquid Cooling and Data Center Water Use separates those mechanisms.

Density changes the building before it changes the headline

A 120 kW rack affects much more than electrical capacity. It concentrates mass, heat and failure consequences.

Structural loading has to cover the rack, contained liquid, busways, manifolds and maintenance routes. The floor must support both installed equipment and the method used to move it into position. Pipework and cable routes compete for space. Leak detection, isolation and drainage have to be designed around the actual topology. Commissioning must test the full chain under representative load or an engineered load-bank strategy.

This can make an apparently suitable brownfield building less attractive than a new-build option. The building may have enough gross floor area and a credible utility connection but lack the structural, hydraulic or distribution geometry needed for the target density.

That does not mean brownfield reuse is impossible. It means the reuse case needs evidence, as set out in Brownfield Data Centers: Reuse Without Assumptions. Lower-density zones, selective strengthening or a different compute mix may produce a better economic result than forcing every hall toward the maximum rack specification.

Average load and design load serve different decisions

Another common error is to replace the design envelope with expected average utilisation too early.

Average demand is essential for energy procurement, operating cost and carbon analysis. It may also support dynamic power management. But protective devices, distribution equipment and cooling response still need a defensible design case. The workload may create fast transitions or concentrated peaks even when the monthly average is lower.

NVIDIA’s dynamic-power documentation is relevant precisely because it treats power as something measured and controlled within an explicit hierarchy. A power cap can create capacity value, but only if the compute workload, control software and facility response have been validated together.

I would separate three cases in every model:

• maximum technical envelope;
• contracted or controlled operating cap;
• expected energy profile over time.

The first protects infrastructure. The second supports commercial and operational commitments. The third informs energy cost. Substituting one for another hides risk rather than removing it.

Where the conclusion does not apply

Not every AI deployment needs a dedicated 120 kW design. Mixed-use facilities can combine conventional cloud, storage and high-density clusters. Some customers may deploy fewer racks with strict caps. Others may prioritise upgrade paths over maximum density on day one.

The argument also changes when a vendor provides a complete, validated system design and the site reproduces its assumptions closely. In that case, the reference architecture can shorten engineering work. It still does not prove the utility schedule, local permit, commercial demand or construction programme.

The point is not that high-density specifications are unreliable. It is that they are local to a boundary.

The decision rule

Before underwriting an AI-ready site, I would require one joined schedule with five columns: rack configuration, IT load, electrical path, cooling path and evidence date. Every MW should state whether it is rack, IT, facility or utility capacity. Every stage should state whether it is designed, ordered, installed, energised, tested or measured.

PowerlandMap supports this comparison by keeping site, power, development and infrastructure evidence in their own scopes rather than collapsing them into one capacity figure. The method is the same one described in From Grid MW to Live IT Capacity: move from headline to deliverable capacity one boundary at a time.

For the 48-rack example, the 5.76 MW rack total is useful. The 7.632 MW illustrative facility demand is more useful. Neither is bankable until the electrical, cooling, operating and utility assumptions are reconciled with the site.

Rack density is a powerful design input. The investable number is the capacity the complete system can deliver, repeatedly and on schedule.