CoreWeave is the most visible example of a category that barely existed a few years ago: the GPU neocloud — a provider whose entire product is accelerated compute, not a general-purpose cloud that happens to rent GPUs. The distinction is not marketing. A hyperscaler is engineered around millions of small, independent, elastic virtual machines; a training cluster is one enormous tightly-coupled job where every node must talk to every other node at line rate for weeks. Those two targets pull hardware, networking, scheduling and contracts in opposite directions. This piece treats the neocloud as a category, with CoreWeave as the worked example: what a specialised provider optimises for, what it deliberately does not build, and how to evaluate one before you sign. Several companies now occupy this space with genuinely different models — they are not interchangeable, and the criteria below are how you tell them apart.
The neocloud thesis — one workload, optimised end to end
A general-purpose cloud sells a very large number of small, loosely-coupled units of compute. Its economics come from statistical multiplexing: thousands of unrelated tenants whose demand peaks do not coincide, packed onto shared hardware and shared network. Almost every design decision follows from that — oversubscribed east-west bandwidth, hypervisor isolation between tenants, instance-granular allocation, on-demand elasticity.
A neocloud inverts those assumptions because it sells one workload shape. Large-model training is a single job spanning hundreds or thousands of GPUs, running a synchronous collective on every step, for weeks without interruption. There is no multiplexing to exploit: the job wants all of the machines, all of the bandwidth, all of the time. So the specialised provider stops trying to be good at everything and optimises the whole stack — fabric, host, storage, scheduler, contract — for that one shape. Everything below is a consequence of that single decision.
The fabric is the product
The clearest technical differentiator is the network. Distributed training is dominated by collectives — all-reduce over gradients, all-gather and reduce-scatter under sharded optimisers — and a collective runs at the speed of its slowest link. Add tensor or expert parallelism and the traffic becomes constant, bulk, all-to-all. On a general-purpose cloud network that traffic pattern meets an oversubscribed spine designed for bursty, mostly north-south tenant traffic, plus a TCP stack in the way.
Neoclouds build the opposite: a dedicated RDMA fabric, typically InfiniBand, arranged as a fat tree with little or no oversubscription at the leaf and spine layers, so bisection bandwidth is close to full. GPUDirect RDMA lets a NIC read and write GPU memory without bouncing through host DRAM, and NCCL is tuned against the real topology rather than a generic default. What matters commercially is that this fabric is capital-intensive, must be designed before the racks land, and cannot be retrofitted — which is precisely why it is the moat.
Bare metal, or virtualisation thin enough not to matter
The second structural difference is what sits between your process and the silicon. Hyperscaler GPU instances are virtual machines, with a hypervisor mediating device access, tenant isolation on the NIC, and an abstraction layer that hides the physical topology from you. For a web service that is exactly right. For a training job it costs you three things: a slice of throughput, visibility into which GPU sits on which PCIe root complex and which NUMA node, and the ability to tune the machine.
Specialised providers lean toward bare metal or a deliberately thin virtualisation layer with passthrough devices. You get the real topology, real NUMA affinity, direct access to NVLink and NVSwitch inside the node, and enough control to pin processes, set NCCL environment variables meaningfully, and profile without a hypervisor blurring the picture. The cost is inherited responsibility: driver versions, firmware, kernel tuning and node health are now at least partly yours, not a detail the platform hides.
Allocation shape — reserved blocks, not on-demand instances
How capacity is handed to you is as important as how fast it is. The cloud default is per-instance, on-demand allocation: ask for machines, get whatever the scheduler has, anywhere in the availability zone. That is useless for training. Sixty-four GPUs scattered across a datacentre are not a cluster; they are sixty-four GPUs with a bad network between them.
The neocloud primitive is a block: a contiguous, topology-aware set of nodes on the same fabric island, allocated together and held for the duration. Hyperscalers have added their own cluster-shaped products in response, which is itself evidence that the ordinary instance model did not fit. Capacity is now planned and committed rather than summoned; scaling is stepwise rather than smooth; and the question you ask a provider is not “what is the instance price” but “what block sizes can you allocate on a single non-blocking fabric domain, and when.”
Contract structure — commitment is the pricing mechanism
Because capacity is physically reserved rather than statistically shared, the commercial model shifts from consumption to commitment. Meaningful access to a specialised provider generally means a term contract for a defined quantity of capacity, not a credit card and an API call. Without quoting numbers — they move constantly — the shape is consistent: longer terms and larger blocks buy better unit economics and better availability, and short-term or on-demand access, where offered, is the expensive exception rather than the default.
This changes who inside your company owns the decision. A hyperscaler GPU bill is an engineering choice made monthly and reversible. A reserved cluster is a finance decision with a duration, and it transfers utilisation risk onto you: you pay for the block whether or not your jobs fill it. That makes cluster utilisation — queue depth, job packing, checkpoint/restart hygiene — a first-order cost lever rather than an operational nicety.
The storage tier that feeds the fabric
A training cluster starves without a storage tier built for it, and this is where object storage quietly fails. Data loading is thousands of workers issuing small random reads across a shared dataset; checkpointing is the opposite — an enormous synchronised burst of writes from every rank at once, during which the GPUs are idle. A general-purpose object store handles neither pattern well at cluster scale.
So neoclouds pair the fabric with a high-performance parallel filesystem, usually NVMe-backed and sourced from the specialist vendors in that space, presented as a POSIX-style shared namespace and reachable over the same RDMA fabric. Where the stack supports it, GPUDirect Storage moves data by DMA from NVMe into GPU memory without a host bounce. Evaluate this tier on read throughput at full cluster width, small-file metadata performance, and how long a full-cluster checkpoint write actually takes — that last number is pure lost GPU time on every save.
The operational surface — Kubernetes-native, not managed-everything
Specialised providers tend to expose a scheduling and orchestration layer rather than a catalogue of managed services. In practice that means Kubernetes-native infrastructure with GPU-aware scheduling: device plugins, topology-aware placement, gang scheduling so a distributed job starts only when every rank has a slot, and integration with the queueing systems ML teams already use.
The gap-filling matters more than the headline. GPU fleets fail in ways CPU fleets do not — a single GPU falling off the bus, ECC errors accumulating, a NIC link flapping, thermal throttling on one node — and in a synchronous collective any one of those stalls the entire job. So the serious providers ship node health checking, automated draining and replacement of bad hardware, fabric-level monitoring, and job-restart integration. This is the least visible and most consequential difference between providers: not who has GPUs, but whose cluster keeps a two-week job alive.
What you give up against a hyperscaler
The specialisation is a genuine trade, and the losses are real. Managed-service breadth is the big one: there is no equivalent of the hyperscaler catalogue — managed relational and NoSQL databases, queues, data warehouses, streaming, serverless functions. Your application tier and your data platform stay where they are, which usually means running across two providers.
Regional coverage is narrower: a handful of datacentres, not dozens of regions, which constrains data-residency stories and edge proximity. Integrated identity and networking is thinner — fine-grained IAM, VPC constructs, private-link plumbing and org-wide policy tooling have to be assembled. And operational maturity in the boring sense — incident history, compliance attestations, quota and support machinery, ecosystem tooling that assumes your provider — sits with the incumbents. If data gravity or compliance already anchors you to a hyperscaler, the cross-cloud data path may cost more than the fabric saves.
How to evaluate a neocloud — the diligence list
The questions that actually separate providers are unglamorous and specific:
| Ask about | What you are really testing |
|---|---|
| Fabric topology | Oversubscription at leaf and spine, largest non-blocking domain, measured all-reduce bandwidth at your block size |
| SLA terms | What the credit covers, whether a degraded node counts as an outage, how a partially failed cluster is treated |
| Eviction policy | Whether reserved capacity can be reclaimed, notice period, what happens to a running job |
| Egress | Cost and bandwidth of data in and checkpoints out — the hidden tax of a two-cloud architecture |
| Support depth | Whether you reach an engineer who can read a fabric counter; time to replace a dead node |
| Storage | Throughput at full cluster width, metadata performance, real checkpoint write time |
Ask for a burn-in window on the real block before the term starts, and run your own collective benchmarks and a multi-day soak. Marketing describes the hardware; only measurement describes the cluster.
Where the model fits
The honest decision rule is about workload shape and time horizon. A specialised provider earns its keep when you are running large, tightly-coupled, long-duration training on a known quantity of capacity, when fabric quality and sustained availability dominate your cost per useful GPU-hour, and when you can commit far enough ahead to plan capacity rather than summon it.
A hyperscaler remains the better answer when your GPU use is bursty and small-scale, when the accelerators live next to a large managed-data footprint you are not moving, when compliance or regional coverage is the binding constraint, or when your team wants the breadth of services more than the last increment of interconnect. Many organisations settle on both — committed specialised capacity for training runs, hyperscaler infrastructure for the application and data tier — and then spend their engineering effort on the seam between them, because that seam is where the cost and the failures accumulate.