A modern data center looks deceptively simple from the road. It may resemble a windowless warehouse surrounded by fences, electrical equipment, cooling machinery, and acres of parking or gravel. Inside, however, it is part power plant, part industrial refrigeration system, part telecommunications exchange, and part automated factory.
The computers are only the most visible layer. Every useful watt delivered to a processor must first pass through utility substations, transformers, switchgear, backup systems, distribution equipment, and power supplies. Nearly every watt then becomes heat that must be moved away from the chips and rejected outdoors. At the same time, thousands of machines must exchange data quickly enough to function as one service—or, in an artificial-intelligence cluster, increasingly as one enormous computer.
That combination makes data centers unusually interesting buildings. Their architecture is dictated less by human comfort than by the needs of machines: uninterrupted electricity, controlled temperature and humidity, immense network capacity, physical security, rapid repair, and predictable failure behavior.
The rise of AI has not made the cloud data center obsolete. It has added a new and much more demanding class of workload. Conventional cloud facilities were designed primarily to operate fleets of comparatively independent servers. AI facilities are increasingly designed around dense, tightly synchronized accelerator clusters whose power and cooling requirements can be an order of magnitude greater at the rack level.
This is the anatomy of those facilities, from the utility connection to the silicon.
The major systems of a modern data center operate as one coordinated machine. 1. What a data center actually is
At its simplest, a data center is a building—or a portion of one—containing computing, storage, and networking equipment supported by dedicated electrical, cooling, fire-protection, security, and control systems.
That definition includes several distinct types of facility:
Enterprise data centers are owned by an organization for its own applications, databases, and internal systems.Colocation facilities rent secured racks, cages, or suites to customers who install their own equipment.Hyperscale cloud data centers are built by companies such as Amazon, Google, Meta, and Microsoft to operate enormous standardized fleets.Edge data centers place smaller amounts of computing closer to users, factories, cellular networks, or other latency-sensitive workloads.High-performance computing and AI data centers concentrate accelerators and high-speed interconnects for scientific computing, model training, and large-scale inference.
A single campus may contain several buildings and hundreds of megawatts of planned electrical capacity. Within a cloud provider's geography, individual facilities may also be grouped into fault-isolated zones. For example, an Amazon Web Services Availability Zone consists of one or more discrete data centers with independent power, networking, and connectivity, while several zones form a Region.1
A useful distinction is between white space and gray space :
White space is the clean, controlled area containing racks, servers, switches, and storage.Gray space contains much of the machinery supporting them: switchgear, batteries, uninterruptible power supplies, pumps, chillers, coolant distribution equipment, and building controls.
In many facilities, the gray space occupies as much attention—and sometimes nearly as much physical area—as the computers.
2. The first model: the enterprise computer room
Before cloud computing became dominant, most organizations treated computing infrastructure as specialized capital equipment. A bank, university, manufacturer, hospital, or government agency might operate one primary data center and one disaster-recovery site. Smaller organizations maintained server rooms in office buildings.
The traditional room often had several recognizable features:
Raised floors used as a plenum for cold air and a pathway for power and data cables.
Perimeter computer-room air conditioners, usually called CRAC units, or chilled-water computer-room air handlers, called CRAH units.
Rows of vendor-specific servers, storage arrays, tape libraries, and network appliances.
Large centralized storage-area networks using Fibre Channel.
Three-tier networks divided into access, aggregation, and core layers.
Redundancy concentrated in expensive individual devices.
Relatively low average rack densities, often only a few kilowatts per rack.
Equipment was frequently purchased around specific business applications. One system might run payroll, another email, another a database, and another a file server. Capacity planning meant predicting future peak demand and buying enough hardware in advance. Because installing additional electrical and cooling capacity was disruptive, rooms were commonly overbuilt and underutilized.
Virtualization improved this model by allowing many software-defined servers to share one physical machine, but the basic facility remained centered on carefully maintained pieces of equipment. A failed enterprise storage array or core switch could be catastrophic, so individual systems were engineered with redundant controllers, power supplies, fans, and paths.
This approach produced durable facilities, but it was expensive and slow to scale. It also encouraged organizations to spend money cooling large rooms full of lightly utilized servers.
3. The cloud-era data center: the warehouse becomes the computer
The hyperscale cloud changed the unit of design. Instead of treating each server as precious, cloud operators treated the entire fleet as the dependable system.
Google began building and operating its own data centers in 2006, while Amazon Web Services launched its first major cloud services during the same period. The Open Compute Project, initiated by Facebook in 2011, later helped spread hyperscale ideas about simplified servers, open rack designs, power distribution, and efficient facilities.2
The cloud-era design had several defining characteristics.
Standardized, replaceable servers
Hyperscalers bought or designed servers by the tens of thousands. A machine was expected to fail eventually. Software detected the failure, shifted work elsewhere, and allowed a technician to replace the server later.
This reduced the need to make every box individually fault-proof. Reliability moved upward into distributed software, replicated storage, load balancing, and geographic architecture.
Horizontal scaling
Traditional systems often scaled up by purchasing a larger server. Cloud systems scaled out by adding more servers. Applications were redesigned to divide work among many machines.
A data hall could therefore be built from repeatable blocks of racks, power distribution, network connections, and cooling capacity. Expansion became a matter of adding another block or another building.
Virtualized and software-defined infrastructure
Customers did not need to know which physical server held a virtual machine, object, or database replica. Compute, storage, and networking became pools allocated through software.
This abstraction enabled high utilization and rapid provisioning. It also allowed operators to perform maintenance without shutting down the customer-facing service, provided workloads were distributed correctly.
Leaf-spine networking
Cloud traffic is dominated by server-to-server, or east-west , communication. The older three-tier network was increasingly replaced by a leaf-spine fabric in which every leaf switch connects to every spine switch. Multiple equal-cost paths provide predictable latency, high aggregate bandwidth, and graceful degradation when a link or switch fails.3
Purpose-built efficiency
Hyperscalers removed features that made sense in small server rooms but added little value at fleet scale. They optimized fan controls, power supplies, rack layouts, airflow, voltage conversion, and software scheduling together.
Power Usage Effectiveness, or PUE , became the best-known facility metric:
PUE = Total facility energy / IT equipment energy
A theoretical PUE of 1.0 would mean that every watt entering the facility reaches IT equipment, with none used by cooling, pumps, lighting, power conversion, or other overhead. Real facilities are always higher. Google reported a fleet-wide average PUE of 1.09 for 2025, meaning approximately nine percent of facility energy was used outside the IT load.4
PUE does not measure whether the computing itself is useful or efficient. A nearly empty server can sit in a facility with an excellent PUE. The cloud improved efficiency partly because it attacked both sides of the equation: facility overhead and server utilization.
4. The AI-era data center: from fleet to factory
Cloud applications can often tolerate differences in timing among servers. A web request can be sent to one of many machines, and a failed machine can be removed from service without interrupting the whole fleet.
Large AI training jobs behave differently. Thousands of accelerators repeatedly exchange model parameters, gradients, activations, and synchronization messages. A delay in one part of the cluster can stall many other devices. The value of the system therefore depends not only on the number of processors, but on how effectively they operate together.
This is why AI infrastructure is often described as an AI factory . The phrase is partly marketing, but it captures a real architectural change: data enters in the form of training examples, prompts, and model state; the facility converts electricity and computation into trained models or generated tokens.
Rack-scale computers
A conventional rack is mainly a cabinet holding separate servers. New AI systems increasingly treat the rack itself as an integrated computer.
NVIDIA's GB200 NVL72, for example, connects 36 Grace CPUs and 72 Blackwell GPUs through an in-rack NVLink fabric. The rack consumes approximately 120 kilowatts and uses liquid cooling for its most power-intensive components.5
At the next scale, racks are joined by InfiniBand or specialized Ethernet fabrics into clusters containing thousands or tens of thousands of accelerators. NVIDIA's current reference architectures divide those clusters into repeatable scalable units , much as cloud operators divide facilities into repeatable infrastructure blocks.6
Density changes the building
In a conventional cloud hall, 10 to 20 kilowatts per rack may be substantial. AI halls increasingly plan for 40, 80, 120 kilowatts or more. Open Compute Project work now contemplates power architectures for racks from 100 kilowatts toward one megawatt, although megawatt racks are a forward-looking design target rather than a common present-day installation.7
At these densities:
Traditional air cooling becomes impractical or inefficient.
Rack power cannot be delivered through ordinary branch circuits and power strips.
Floors and racks must support heavier integrated systems and coolant manifolds.
Cable routing becomes a major engineering constraint.
Rapid electrical load changes can affect generators, UPS systems, and the grid connection.
A single rack can represent millions of dollars of equipment and a large fraction of a cluster's productivity.
The building is no longer simply a place where computers are installed. The mechanical and electrical plant must be co-designed with a specific computing architecture.
5. Choosing the site
The first critical component of a data center is not a server. It is the site.
A data-center campus brings buildings, utility infrastructure, cooling equipment, and network connections together on one site. Electricity
Modern hyperscale campuses are often limited by available electrical capacity rather than land. The developer must determine whether the local utility can provide the required megawatts, how quickly transmission and substations can be built, whether the grid can tolerate the load, and what the electricity will cost over decades.
A 2024 Department of Energy advisory noted connection requests for hyperscale facilities in the 300-to-1,000-megawatt range and warned that one-to-three-year requested lead times could exceed the pace at which local grids can add capacity.8
Developers increasingly evaluate:
New utility substations and transmission lines.
On-site generation or microgrids.
Battery energy storage.
Long-term wind, solar, nuclear, geothermal, or hydroelectric contracts.
Demand response and the ability to shift flexible computing.
Former industrial or power-plant sites with existing grid connections.
AI training can sometimes be scheduled around electricity availability, but customer-facing inference must respond when users make requests. A mixed facility therefore contains both flexible and inflexible loads.
Fiber and latency
A data center needs multiple physically diverse fiber routes. Cloud facilities must connect to customers, other regions, internet exchanges, content networks, and private backbones. AI campuses also need enormous internal bandwidth between buildings.
Fiber route diversity matters as much as raw capacity. Two cables entering opposite sides of a building are not truly independent if they share the same bridge, conduit, railroad crossing, or upstream carrier facility.
Water and climate
Cool climates can increase the number of hours when outdoor air or water can provide “free cooling.” Dry climates may improve evaporative cooling performance but create water-supply conflicts. Humid climates complicate evaporative systems and heat rejection.
There is no universally best cooling method. A design that minimizes electricity may consume more water; a dry closed-loop system may save water but use more fan or compressor energy during hot weather.
Risk, workforce, and community
Floodplains, tornadoes, hurricanes, wildfire smoke, seismic activity, extreme heat, and ice all affect design. So do access to electricians, controls technicians, network specialists, security staff, and emergency services.
Tax incentives can heavily influence siting, but the long-term relationship with the surrounding community increasingly depends on transparent plans for power, water, noise, backup-generator testing, land use, and infrastructure costs.
6. The electrical anatomy: from the grid to the chip
A data center's power system is a chain. A simplified path looks like this:
Utility transmission or distribution → substation → medium-voltage switchgear → transformers → low-voltage switchgear → UPS or battery system → power distribution units or busway → rack power shelves or rack PDUs → server power supplies → voltage regulators → chips
Power passes through utility and on-site infrastructure before it reaches computing equipment. Every conversion loses some energy and creates heat. Every component can also fail, so designers decide where redundancy is necessary and how maintenance can occur without interrupting the IT load.
Utility service and substations
Large campuses may receive power at transmission or sub-transmission voltages and operate their own substations. Transformers reduce that voltage in stages to levels usable by the building and IT equipment.
AI density is encouraging designers to keep voltage higher for longer. Delivering the same power at a higher voltage requires less current, which reduces conductor size and resistive losses. New rack-level designs are exploring higher-voltage direct current instead of distributing extremely high current at 48 volts.
Switchgear and fault protection
Switchgear contains breakers, relays, disconnects, metering, and buswork. It isolates faults and allows sections of the electrical system to be maintained.
A data center is carefully divided into electrical fault domains. A failed breaker or transformer should affect a known and limited portion of the load rather than cascade across the building.
Uninterruptible power supplies
A UPS bridges the gap between utility failure and backup generation. Traditional systems use batteries, while some installations use flywheels or other storage technologies.
The UPS also conditions power, keeping voltage and frequency within acceptable limits. At hyperscale, batteries may perform additional functions such as smoothing sudden AI load changes, supporting generator ramp-up, or providing grid services.
Backup generation
Diesel generators remain common because they can start quickly, store energy on-site, and operate independently of pipeline infrastructure. Natural-gas engines, turbines, fuel cells, and large batteries are also being considered or deployed in some designs.
Generators are not instant. Batteries support the load while engines start, stabilize, and accept it. Fuel storage and refueling plans determine how long a site can operate during an extended outage.
Distribution to the rack
Older rooms often used floor-mounted power distribution units feeding circuits under a raised floor. Modern halls increasingly use overhead busways, which allow tap boxes to be added or moved as rack layouts change.
AI racks may contain dedicated power shelves that convert facility power into high-current DC for compute trays and switches. Dual power paths are common, but the exact design depends on whether redundancy is provided in the rack, row, hall, or software layer.
N, N+1, 2N, and concurrent maintainability
Data-center engineers describe capacity and redundancy with compact notation:
N is the capacity required to support the load.N+1 adds one spare component or module.2N provides two complete independent systems.2N+1 provides two complete systems plus additional reserve.
More redundancy does not automatically mean better design. It increases cost, space, materials, complexity, and sometimes energy losses. The appropriate topology depends on the consequences of failure and the ability of the software to move work elsewhere.
The Uptime Institute's Tier system evaluates facility topology from basic capacity through fault-tolerant infrastructure. Tier III is associated with concurrent maintainability, while Tier IV adds fault tolerance. The standard defines performance objectives rather than prescribing one exact technology.9
7. Cooling: every watt becomes heat
Almost all electricity consumed by computing equipment eventually becomes heat inside the building. A 120-kilowatt rack is therefore also a 120-kilowatt heater.
Cooling is a chain similar to power:
Chip → heat spreader or cold plate → air or liquid loop → row or facility loop → heat exchanger or chiller → cooling tower, dry cooler, or outdoor air
Hot-aisle and cold-aisle separation keeps server exhaust from mixing with cool supply air. The engineering problem is not to make the building cold. It is to keep each component within its allowable temperature while using as little energy and water as practical.
Air cooling
Air-cooled servers use heat sinks and fans to transfer heat from chips to air. Facility fans then move cold supply air to server inlets and return hot exhaust air to cooling equipment.
Key methods include:
Hot-aisle/cold-aisle layout: Server fronts face fronts and backs face backs.Containment: Doors, roofs, or partitions prevent hot exhaust from mixing with cold supply air.CRAC units: Use refrigerant compressors within or near the computer room.CRAH units: Use chilled water produced by a central plant.Economization: Uses favorable outdoor conditions to reduce compressor operation.Direct or indirect evaporative cooling: Uses water evaporation to cool air, with different degrees of separation between outdoor and data-hall air.
Raised floors are no longer mandatory. Many modern facilities distribute air and cables overhead or use slab floors with contained aisles.
ASHRAE's thermal guidance has gradually allowed wider operating envelopes than early computer rooms, enabling higher supply temperatures and more economizer hours without sacrificing equipment reliability.10
Rear-door heat exchangers
A rear-door heat exchanger replaces or supplements the back door of a rack with a liquid-cooled coil. Hot server exhaust passes through the coil before entering the room.
This can increase the density supported in an otherwise air-cooled hall. It is especially useful for retrofits because servers remain conventionally air cooled, although water or coolant must be brought to every equipped rack.
Direct-to-chip liquid cooling
In direct liquid cooling, cold plates contact CPUs, GPUs, memory, or other hot components. A coolant loop carries heat to a coolant distribution unit, or CDU , which isolates and controls the technology-cooling loop and transfers heat to the facility-water loop.
Liquid has far greater volumetric heat capacity than air. It can remove intense heat with smaller flow passages and lower fan energy. It also allows warmer coolant temperatures, potentially eliminating compressor-based chilling during much of the year.
Many current AI systems are hybrid cooled : liquid removes most of the GPU and CPU heat, while air still cools power supplies, memory, network equipment, and other components.
Microsoft has begun deploying closed-loop, chip-level cooling designs intended to eliminate evaporative water use for cooling at new AI-optimized facilities.11
Immersion cooling
Immersion systems place servers in a nonconductive dielectric fluid.
In single-phase immersion , the liquid remains in liquid form and is pumped through a heat exchanger.
In two-phase immersion , heat boils the fluid; vapor condenses on a cooled surface and returns to the tank.
Immersion can remove heat efficiently and eliminate server fans, but it changes maintenance, material compatibility, cabling, safety procedures, and hardware design. Microsoft has operated two-phase immersion cooling in a production environment, though direct-to-chip systems currently appear to have broader momentum for mainstream AI racks.12
Microfluidic and future cooling
Research is moving the coolant even closer to the heat. Microsoft reported testing microfluidic channels etched into the back of silicon, with cooling performance up to three times better than conventional cold plates in its tested configurations.13
Other development areas include:
Two-phase cold plates.
Refrigerant-based direct cooling.
Manifolded rack-scale systems.
Waste-heat recovery.
Higher-temperature coolant loops that can reject heat without chillers.
Standardized quick-disconnects and leak-detection systems.
The limiting factor is increasingly the thermal path inside the package rather than the temperature of the room.
8. The compute hardware
Standardized racks organize servers, network equipment, power distribution, and cabling into repeatable units. General-purpose cloud servers
A conventional cloud server usually contains one or two CPU sockets, large amounts of DRAM, local solid-state storage, network interfaces, and redundant power supplies. The CPU is optimized for flexible, branching workloads: databases, virtual machines, web services, control planes, and business applications.
Hyperscalers often design custom motherboards, accelerators, security chips, storage devices, and server management systems. What appears to the customer as a generic virtual machine may run on highly specialized provider hardware.
Accelerated AI servers
AI systems add GPUs or purpose-built accelerators. Compared with CPUs, these devices devote much more silicon to highly parallel mathematical operations. They are paired with high-bandwidth memory, or HBM , which feeds the processors far faster than ordinary server memory.
An AI node may contain:
Several GPUs or accelerators.
CPUs responsible for orchestration and preprocessing.
HBM attached to each accelerator.
High-speed links among accelerators within the node.
Network interfaces connecting the node to the cluster fabric.
Local NVMe storage for caching and temporary data.
Baseboard management controllers for out-of-band administration.
The accelerator is only useful when it is fed. Insufficient memory bandwidth, network bandwidth, storage throughput, or power can leave extremely expensive silicon idle.
Scale-up and scale-out
AI infrastructure uses two related forms of expansion:
Scale-up connects accelerators with very high-bandwidth links so they behave like one larger machine. NVLink domains are an example.Scale-out connects many nodes or racks through a network fabric.
Scale-up offers the tightest coupling but is constrained by physical packaging, cable reach, switch design, and cost. Scale-out reaches far larger clusters but places more pressure on the network and distributed software.
9. Storage: feeding the accelerators
Traditional enterprise storage emphasized transactional reliability, centralized management, snapshots, and predictable latency. Cloud storage expanded that model with distributed object stores, replicated block storage, and enormous data lakes.
AI adds another demanding pattern. Training jobs may read massive datasets, generate checkpoints containing model state, and restart after a failure. Thousands of processors may request data simultaneously.
A modern AI storage hierarchy may include:
Local HBM for the data actively used by each accelerator.System DRAM for CPU-side processing and caching.Local NVMe for temporary files, dataset caches, and fast checkpoints.High-performance shared storage using a parallel file system or scale-out storage appliance.Object storage or a data lake holding the larger corpus, archives, and inactive datasets.Backup or archival storage , sometimes including tape for low-cost long-term retention.
NVIDIA's SuperPOD designs distinguish high-performance storage from user storage. The performance tier is expected to support parallel reads and writes, high throughput, high IOPS, metadata operations, and native high-speed networking.14
The storage system must be balanced with the compute system. Adding accelerators without expanding storage can reduce overall productivity because processors wait for data. Conversely, building a storage system for theoretical peak throughput can be wasteful if software cannot issue enough parallel I/O.
Checkpointing illustrates the scale problem. A large training run periodically writes its state so it can resume after failure. If every node writes at once, the storage fabric experiences an enormous burst. The checkpoint must complete quickly enough that the cluster returns to productive work, but reliably enough that it can actually restore the job.
10. Networking: the nervous system
A data center contains several logically separate networks.
Front-end or service network
This carries customer requests, application traffic, internet connectivity, and communication with other facilities. Conventional Ethernet and IP dominate.
Back-end compute fabric
This connects accelerators during distributed training or inference. It requires very high bandwidth, low latency, low packet loss, and predictable behavior under congestion.
InfiniBand has been widely used for high-performance AI clusters because it was designed for remote direct memory access and tightly coupled computing. Ethernet alternatives use technologies such as RoCE, or RDMA over Converged Ethernet, along with careful congestion control and loss management.
The Ultra Ethernet Consortium released its 1.0 specification in 2025 to create an interoperable Ethernet-based communications stack optimized for AI and high-performance computing.15
Storage network
Storage traffic may use a dedicated InfiniBand or Ethernet fabric. Current NVIDIA reference designs call for more than 40 gigabytes per second of storage I/O per compute node in some configurations.16
Management network
A separate in-band management network handles operating systems and cluster services, while an out-of-band network reaches baseboard management controllers even when a server's main operating system is down.
Inside the rack
The network hierarchy now extends all the way into the machine. An AI rack may contain:
Accelerator-to-accelerator links.
In-rack NVLink switches.
Network interface cards or data-processing units.
Top-of-rack Ethernet or InfiniBand switches.
Optical transceivers and copper links.
Separate control and management connections.
Cables are no longer an afterthought. Thousands of high-speed links create challenges involving bend radius, airflow, optical power, connector cleanliness, routing distance, labeling, installation sequence, and replacement access. At cluster scale, the cable plant is one of the defining physical features of the computer.
11. Fire protection, physical security, and controls
Data centers are occupied by relatively few people, but they require extensive protection systems.
Fire detection and suppression
Very early smoke-detection systems continuously sample air and can detect combustion before visible smoke develops. Facilities may use pre-action sprinkler systems, clean-agent suppression in selected rooms, and carefully zoned alarms.
Lithium-ion batteries introduce additional detection, separation, ventilation, and emergency-response requirements. Fuel storage and generators have their own fire codes and containment systems.
Physical security
Security commonly includes perimeter fencing, vehicle controls, cameras, guards, anti-tailgating vestibules, biometric access, compartmentalized permissions, and detailed audit logs. Customer equipment in colocation facilities may be separated into cages or private suites.
Physical access does not automatically grant data access. Modern systems also depend on encryption, hardware roots of trust, secure boot, key management, and destruction procedures for failed storage devices.
Building and infrastructure controls
A building-management system monitors mechanical equipment, while an electrical power-monitoring system tracks breakers, transformers, UPS units, and distribution. Data-center infrastructure management software may combine power, cooling, rack, cable, and capacity information.
AI facilities add cluster-scale telemetry: temperatures, coolant pressures, pump status, link errors, GPU health, workload performance, and power transients. The operations team must correlate a slowing training job with anything from a failed optical transceiver to an obstructed filter or unstable coolant loop.
Operations teams use centralized telemetry to monitor computing, network, power, cooling, and security systems. 12. Reliability is designed in layers
No data center is failure-proof. The objective is to prevent ordinary failures from becoming service outages.
Reliability exists at several layers:
Component: Redundant fans, power supplies, pumps, links, and controllers.Rack or row: Dual power paths, redundant switches, isolated coolant branches.Data hall: Separate electrical and mechanical zones.Building: Multiple utility paths, generators, and independent infrastructure blocks.Campus: Workloads distributed among buildings.Region: Services replicated among fault-isolated zones.Geography: Critical data and applications replicated between regions.
The cloud made software resilience more important than perfection in any one building. AI complicates that philosophy. A training job may survive hardware failure through checkpointing and workload restart, but the lost time can be extremely expensive. As clusters grow, failures become statistically routine, so automated detection, isolation, repair, and restart are part of the production system.
Commissioning is therefore critical. Before a facility enters service, teams test individual components, integrated systems, failure modes, control sequences, and emergency operations. They deliberately remove utility power, disable pumps, trip breakers, fail network paths, and verify that the building reacts as designed.
13. Efficiency, water, and environmental tradeoffs
The United States Department of Energy reported that data centers used about 176 terawatt-hours of electricity in 2023, approximately 4.4 percent of total U.S. electricity consumption. It projected a range of roughly 325 to 580 terawatt-hours by 2028, or about 6.7 to 12 percent of U.S. electricity use.17
Those estimates include uncertainty about AI adoption, hardware efficiency, utilization, and construction. They nevertheless show why data centers have become an important power-system issue.
Metrics beyond PUE
PUE measures facility overhead, but a more complete assessment may include:
WUE — Water Usage Effectiveness: Water consumed relative to IT energy.CUE — Carbon Usage Effectiveness: Carbon emissions associated with data-center energy.WUI — Water Usage Impact: Attempts to account for local water scarcity and impact, not merely volume.IT utilization: The share of installed computing capacity doing useful work.Compute or tokens per watt: Work produced per unit of energy.Hardware utilization and lifetime: Whether expensive equipment is used long enough to justify its manufacturing footprint.
ASHRAE's 2026 AI Data Center Energy Performance Framework recommends evaluating multiple energy, water, carbon, and utilization metrics rather than treating one number as a complete score.18
Water-energy tradeoffs
Evaporative cooling can reduce electrical consumption by using water instead of compressors. Dry coolers can reduce water consumption but may require more fan power or higher coolant temperatures. Chillers provide precise cooling but add capital cost and energy use.
The correct choice depends on climate, water source, grid conditions, equipment temperature limits, and community priorities.
Carbon-free energy is not the same as constant local supply
Annual renewable-energy purchases can match a facility's yearly consumption while the data center still uses fossil-heavy grid power during certain hours. More ambitious strategies seek hourly matching with carbon-free generation, energy storage, geographic workload shifting, or firm resources such as nuclear, hydroelectric, or geothermal power.
Heat reuse
Warm-water liquid cooling makes waste heat more usable. Heat can support district heating, greenhouses, industrial processes, or nearby buildings. The difficulty is economic and geographic: heat is costly to transport, demand varies seasonally, and the data center and heat user must be built close together.
Google's Hamina, Finland, data center, for example, combines seawater cooling with an off-site heat-recovery project.19
14. How data centers are built now
The pace of AI deployment has made construction speed a competitive metric often called time to power or time to first token .
Modular infrastructure
Electrical rooms, pump skids, cooling modules, and even complete data halls can be fabricated off-site, tested in a factory, and delivered as repeatable units. Modular construction reduces field labor and makes designs easier to replicate across campuses.
Standardized blocks
A campus may be planned as a series of identical power-and-cooling blocks, each supporting a known number of racks. Standardization simplifies procurement, commissioning, spare parts, training, and software integration.
Phased construction
Developers rarely install the entire planned campus at once. They build the utility and site backbone, then add substations, halls, cooling plants, and IT equipment in phases. This reduces the amount of idle capital and allows later phases to adopt newer hardware.
Retrofitting cloud facilities for AI
Many cloud data centers were not designed to deliver liquid coolant or 100 kilowatts to a rack. Retrofitting may involve:
Adding CDUs and secondary coolant loops.
Reinforcing power distribution.
Replacing rack PDUs and busway taps.
Reserving high-density zones rather than converting the entire hall.
Installing rear-door heat exchangers.
Redistributing servers across more racks.
Building adjacent AI-specific halls while retaining older halls for general cloud workloads.
Microsoft has described using liquid-cooling heat-exchanger units to introduce direct-to-chip cooling into legacy air-cooled facilities.20
A successful retrofit depends on more than total building capacity. A facility may have enough megawatts overall but be unable to deliver them to the correct rows, reject the concentrated heat, or route the required network cabling.
15. What makes an AI data center fundamentally different
The distinction is not simply “servers versus GPUs.” Several deeper changes occur at once.
The rack becomes an engineered product
Cloud operators could mix many ordinary servers within standard racks. AI vendors increasingly deliver integrated racks containing compute trays, fabric switches, power shelves, manifolds, controls, and prescribed cabling.
The network becomes part of the computation
In a cloud service, the network connects computers. In large AI training, communication performance directly determines how much useful computation the accelerators can complete. A congested or failed link can reduce the productivity of an entire job.
Power is a computational resource
Schedulers historically allocated CPU, memory, and storage. AI operators increasingly treat electrical power as another schedulable constraint. Cluster software may cap power, stagger workload starts, smooth transient demand, or select jobs according to available capacity.
Cooling reaches the silicon
The facility cooling system once ended at the room air. It now extends through pipes, manifolds, quick-disconnects, cold plates, and perhaps eventually channels inside the package or silicon.
Hardware generations are faster than building generations
A data-center shell may operate for decades. AI hardware can become economically obsolete within a few years. Designers must build long-lived substations, pipes, and structures around short-lived and rapidly changing computing equipment.
This creates a difficult question: how much should be optimized for the current rack, and how much should be reserved for the next one?
16. The most interesting design tensions
Data centers are defined by tradeoffs rather than a single ideal design.
Efficiency versus resilience
Redundant equipment may run at low load and lower efficiency. Eliminating redundancy improves utilization but increases the consequence of failure.
Density versus flexibility
Dense racks use floor area efficiently but concentrate heat, power, weight, and failure impact. Lower density can fit more facilities but requires more cabling, land, and infrastructure.
Water versus electricity
Evaporative systems may save electricity while consuming water. Dry systems may save water while increasing electrical demand during hot periods.
Standardization versus rapid innovation
Repeatable designs reduce cost and risk. AI hardware changes quickly enough that rigid standards may become obsolete before a campus is completed.
Local reliability versus geographic resilience
A highly redundant building is expensive. A service spread across several less-redundant buildings may achieve better overall availability—if its software can tolerate the distribution.
Maximum performance versus useful output
The fastest individual accelerator is not necessarily the most economical system. The relevant measures are completed training runs, useful inference, tokens, scientific results, or customer transactions per dollar, watt, gallon, and hour.
Conclusion
A data center is best understood as a complete energy-and-information machine.
The cloud era transformed it from a room of valuable individual computers into a warehouse-scale computer built from standardized, replaceable parts. The AI era is transforming it again. Dense accelerator systems, rack-scale interconnects, high-performance storage, liquid cooling, and massive electrical loads are turning portions of the data center into something closer to an industrial supercomputer.
Yet the basic anatomy remains recognizable. Electricity arrives, is conditioned and distributed, becomes computation and heat, and leaves as data, economic value, and thermal energy. Networks bind the machines together. Storage keeps their state. Cooling protects their operation. Redundancy limits the damage when components fail. Software determines whether the physical resources are used efficiently.
The remarkable feature of the modern data center is not any single server or cooling technology. It is the coordination of thousands of electrical, mechanical, optical, and digital components into a system that operates continuously—and usually invisibly—behind nearly every online service.
Sources and further reading
1 Amazon Web Services, AWS Regions and Availability Zones .
2 Open Compute Project, Delivering an Open Data Center Ecosystem for AI ; Google, The Dalles, Oregon data center .
3 Cisco, Massively scalable data center network fabric .
4 Google, Power Usage Effectiveness .
5 NVIDIA, DGX GB Rack Scale Systems User Guide ; NVIDIA, GB200 NVL72 .
6 NVIDIA, DGX SuperPOD reference architecture .
7 Open Compute Project, Realizing the Open Data Center Ecosystem Vision .
8 U.S. Department of Energy, Recommendations on Powering Artificial Intelligence and Data Center Infrastructure .
9 Uptime Institute, Tier Classification System .
10 ASHRAE, Data Center Resource Page and Datacom Series ; ASHRAE, 2021 Equipment Thermal Guidelines reference card .
11 Microsoft, Next-generation datacenters consume zero water for cooling .
12 Microsoft, To cool datacenter servers, Microsoft turns to boiling liquid .
13 Microsoft, A microfluidics breakthrough to cool AI chips .
14 NVIDIA, DGX SuperPOD storage requirements ; NVIDIA, DGX SuperPOD administration overview .
15 Ultra Ethernet Consortium, UEC Specification 1.0 announcement .
16 NVIDIA, DGX SuperPOD network fabrics .
17 U.S. Department of Energy, DOE Releases New Report Evaluating Increase in Electricity Demand from Data Centers .
18 ASHRAE, NEMA, and Pacific Northwest National Laboratory, AI Data Center Energy Performance Framework: Energy and Thermal Efficiency .
19 Google, Hamina, Finland data center .
20 Microsoft, Liquid cooling in air-cooled data centers on Azure .
Additional technical references
U.S. Department of Energy and National Renewable Energy Laboratory, Best Practices Guide for Energy-Efficient Data Center Design .
ASHRAE, NEMA, and Pacific Northwest National Laboratory, AI Data Center Energy Performance Framework .
NVIDIA, DGX SuperPOD Architecture .
NVIDIA, DGX SuperPOD Storage Architecture .
Open Compute Project, Cooling Environments .
Open Compute Project, Open Data Centers for AI .
Google, Operating sustainably .
Amazon Web Services, AWS Global Infrastructure .
