Training Here, Inference There: Building a Location Strategy Around AI Workloads

GPUs do not operate in isolation. Their effective performance depends on the infrastructure around them: power delivery, cooling, east-west networking, storage throughput, external connectivity and physical layout.

This becomes increasingly important as AI systems move from individual GPU servers towards rack-scale infrastructure. Modern systems such as NVIDIA GB300 NVL72 can require well above 100 kW per rack and depend on liquid cooling and high-performance network fabrics.

For a neocloud, site selection must account for more than available space for GPU servers. The location needs enough power, cooling, network capacity and room to expand to support the AI workloads planned for it. A poorly matched facility can restrict expansion, complicate fabric design, create cooling limitations or delay the commissioning of expensive GPU assets.

A GPU becomes commercially productive only when the datacenter infrastructure around it supports the workload it is intended to serve.

For an AI Factory, Location Strategy Begins with Megawatts

Traditional colocation discussions often begin with rack counts. Large AI deployments increasingly need to begin with power. A Neocloud planning hundreds or thousands of GPUs needs to understand not only what power is available today, but also what can be committed for future phases.

A 1 MW deployment can become 5 MW or 10 MW as customer demand grows. The site therefore needs a credible expansion path without forcing the operator to redesign or relocate the cluster every time capacity increases.

The key questions include:

• How much utility power is available and when can it be delivered?
• Can capacity expand in contiguous phases?
• What rack densities can the facility sustain?
• Can it support the required liquid-cooling architecture?
• Is there space for future GPU, network and storage expansion?
• Can connectivity scale alongside compute capacity?

For a Neocloud, time-to-power becomes time-to-revenue.

Large-Scale Training Favours Concentrated Infrastructure

Large training workloads require many accelerators to operate as one computational system. That means GPU performance alone is not enough. Training depends on high-bandwidth east-west networking, high-performance storage, consistent server configurations, sustained power and effective cooling.

Weakness in any one of these layers can reduce effective cluster output. Training-oriented locations should therefore prioritise concentrated scale: contiguous high-density space, large power blocks, liquid cooling, predictable network topology and room for storage and fabric expansion.

Physical layout also matters. Cluster expansion across distant halls can increase fiber paths, optics requirements, cabling complexity and operational effort. Where possible, large AI factories should be designed with planned expansion zones from the beginning.

The commercial metric is not simply installed GPU count. It is the amount of reliable, usable compute that the complete AI factory can deliver.

Training Capacity and Colocation Must Be Designed Together

For a Neocloud, the colocation provider becomes part of the AI infrastructure architecture. A training location should therefore be qualified across several dimensions.

• Power and electrical infrastructure: The site should provide committed power with a credible delivery timeline and a roadmap for future expansion.
• High-density cooling: The facility should support the cooling architecture required by the selected GPU platform, including direct-to-chip liquid cooling where necessary.
• Cluster layout: Rack placement, fibre pathways, cable distances, switch locations and expansion zones should be considered together with the logical fabric design.
• Storage placement: High-performance storage requires its own power, space and network connectivity and should be planned together with the GPU cluster.
• Deployment operations: Staging, equipment movement, installation support, secure access and remote hands can materially affect deployment velocity.

At Neocloud scale, operational readiness is part of time-to-market.

Inference Has a Different Location Equation

Inference does not have one deployment model. Some workloads require very low response latency. Others prioritise throughput and infrastructure economics. Latency-sensitive inference may benefit from proximity to users, enterprise applications, databases, vector stores, customer networks and cloud environments.

For these workloads, a well-connected metro datacenter can be strategically valuable even if it does not host the Neocloud’s largest GPU cluster. Important considerations include carrier diversity, cloud connectivity, private interconnects, route quality and latency to target customers.

However, not all inference needs to be distributed. High-throughput, batch or less latency-sensitive inference can remain concentrated in larger AI factories where infrastructure economics are more favourable.

Large-scale training tends to favour concentrated infrastructure. Latency-sensitive inference may benefit from placement closer to users, applications and data.

Data Gravity and Sovereignty Influence Placement

Enterprise data cannot always be moved freely or efficiently. Large datasets may sit across private datacenters, public clouds, databases and regulated environments. Moving them repeatedly to distant GPU clusters can introduce network cost, transfer time, security dependencies and operational complexity.

In some cases, placing selected GPU capacity closer to the data may be more efficient than continuously moving the data towards compute. Sovereign AI introduces additional requirements. Depending on the customer and jurisdiction, these may include where data and models are stored, where processing occurs, who administers the environment and which legal jurisdiction governs it.

A centralized AI factory may therefore be technically efficient but unsuitable for every regulated or nationally sensitive workload.

Every Location Should Have a Defined Role

A mature Neocloud does not need identical GPU infrastructure in every datacenter.

• Core AI factories can provide concentrated power, liquid cooling and large-scale cluster capacity for training, fine-tuning and high-throughput inference.
• Metro locations can position inference closer to enterprise applications, users and network ecosystems.
• Regional or sovereign locations can support data-residency, jurisdictional and regulatory requirements.

These roles are not rigid. Large AI factories can run inference, and metro sites can run smaller training environments. The purpose is simply to place infrastructure according to the dominant technical and commercial requirements of the workload.