Interested in our v5 hardware?
v5 hardware is currently available for deployment in our Ashburn, VA data center and can be ordered directly through our Bare Metal and Private Cloud catalogs. For deployments in other data center locations or for custom configurations, please contact our team.
The v5 generation jump moves more system behavior through wider links than through faster clocks, and the pipe is only as wide as you populate it.
The v5 line (Intel Xeon 6, Granite Rapids, Intel 3) is easy to read as a clocks-and-cores refresh. That is definitely interesting. But a change that helps really move system behavior is bandwidth. The private network fabric doubled from 20 Gbps (two 10 Gbps links) to 40 Gbps (four 10 Gbps links). Memory moved to DDR5-6400, lifting per-socket bandwidth to roughly 410 GB/s from the 307 to 333 GB/s a v4 socket delivered at DDR5-4800 or 5200. The socket interconnect went to UPI at 24 GT/s on up to four links, and each processor now exposes 88 PCIe 5.0 lanes instead of 80.
For a hyper-converged cloud core, where the control plane, compute, and Ceph OSDs share three identical nodes, those links are not specs on a sheet. They are the same wires that carry replication writes, recovery reads, and east-west VM traffic, often at the same time. Widening them, not raising the clock, is what lifts the consolidation ceiling and shortens the windows when the cluster is moving data. The honest part of the story is where the pipe does not widen on its own: two of the three tiers ship with half their memory channels populated, and the NVMe drive interface did not change at all.
Key Takeaways
- The private fabric doubled to 40 Gbps, and it is unmetered. v5 nodes carry four 10 Gbps private links versus two on v4. Per OpenMetal’s networking model, private traffic between servers is not billed, so the replication and recovery storms the wider pipe enables do not show up as transfer charges.
- Memory bandwidth rose to about 410 GB/s per socket. DDR5-6400 across 8 channels lifts per-socket bandwidth roughly 23 to 33 percent over v4’s DDR5-4800/5200, which matters most for memory-bound consolidation and NUMA-spanning guests.
- Medium and Large v5 ship at half their memory bandwidth. Both populate 8 of 16 DIMM slots, which is 4 of 8 channels per socket. Until the open channels are filled, the platform runs near half its DDR5-6400 peak. XL v5 ships full-channel at one DIMM per channel and does not carry this caveat.
- The lane budget grew to 88 PCIe 5.0 lanes per socket. That is 176 lanes per node feeding NVMe and NICs at full width, with UPI at 24 GT/s (up to 4 links) widening the cross-socket path that NUMA-remote access depends on.
- The drive interface did not move. The Micron 7500 MAX is PCIe Gen4 by interface (per the Micron 7500 Tech Prod Spec). Every bandwidth gain in v5 is upstream of the disk, so the place the bottleneck lands shifts accordingly.
| Subsystem | v4 | v5 | Change |
|---|---|---|---|
| Private network fabric | 20 Gbps (2x 10 Gbps) | 40 Gbps (4x 10 Gbps) | 2x |
| Memory bandwidth per socket | ~307 to 333 GB/s (DDR5-4800/5200) | ~410 GB/s (DDR5-6400) | +23 to 33% |
| Socket interconnect (UPI) | 20 GT/s, 3 links | 24 GT/s, up to 4 links | wider and faster |
| PCIe lanes per socket | 80 (PCIe 5.0) | 88 (PCIe 5.0) | +8 lanes |
| NVMe drive interface | PCIe Gen4 x4 | PCIe Gen4 x4 | unchanged |
The private fabric doubled, and that changes how recovery feels
On v4, each node carried two 10 Gbps private links, 20 Gbps total, and that fabric handled everything internal: Ceph replication, Kubernetes service traffic, live migration, and general east-west patterns. v5 doubles it to four 10 Gbps links, 40 Gbps total. The headline number is simple. The consequence is not.
In a hyper-converged core, the replication path is never idle for long. Every write to a 3x replicated pool fans out to peer OSDs over that private fabric. When a drive or a node drops, recovery and backfill read and rewrite objects across the surviving members, and that traffic competes with live client I/O on the same links. On a 20 Gbps fabric, a heavy recovery is something you schedule around, because backfill bandwidth and client tail latency trade against each other directly. Doubling the fabric to 40 Gbps does not make recovery free, but it widens the headroom so that backfill and steady-state client traffic are less likely to collide at line rate. Rebalance windows compress, and the blast radius of a node loss is shorter in wall-clock terms.
The same headroom is what raises the consolidation ceiling. Denser VM packing means more east-west chatter per node. The extra 20 Gbps is the margin that keeps that density from saturating the replication path during a backup window or a migration wave.
DDR5-6400 lifts memory bandwidth, if you fill the channels
Granite Rapids runs 8 memory channels per socket at DDR5-6400. Fully populated at one DIMM per channel, that is roughly 410 GB/s of per-socket memory bandwidth, against about 333 GB/s for a v4 Large at DDR5-5200 and about 307 GB/s for a v4 XL at DDR5-4800. For memory-bandwidth-bound workloads, in-memory databases, analytics, and dense multi-tenant compute, that 23 to 33 percent lift is the difference between cores that stall on memory and cores that stay fed.
Here is the catch an architect has to plan for. Medium v5 and Large v5 ship with 8 of 16 DIMM slots populated, which is 4 of 8 channels per socket. This can be upgraded, but at the default population the node runs close to half the platform’s DDR5-6400 peak memory bandwidth, because half the channels are dark. This is the same physical fact that governs TDX eligibility on these tiers: the platform wants every channel’s first slot filled before it runs at full width. XL v5 ships full-channel at one DIMM per channel, so it reaches the platform’s memory bandwidth out of the box and does not carry this caveat. If a Medium or Large v5 workload is memory-bandwidth sensitive, the channel population is a sizing decision, not a footnote. The pipe is only as wide as you populate it.
The lane budget and the interconnect grew with it
Bandwidth into and across the box matters as much as bandwidth to memory. Each v5 socket exposes 88 PCIe 5.0 lanes, up from 80 on v4, which is 176 lanes per dual-socket node. That lane budget is what lets the all-NVMe drive bays and the NICs run at full width without contending for lanes, the same design principle that keeps the boot pool isolated from the OSD data path. More lanes is not a benchmark number; it is the freedom to attach drives and network without robbing one to feed the other.
The socket interconnect moved too. v5 runs UPI at 24 GT/s on up to four links (the top tier), against 20 GT/s on three links for v4. Any workload that spans both sockets pays for cross-socket memory access over UPI, and a wider, faster interconnect lowers the penalty for NUMA-remote traffic. For a scheduler packing guests across two sockets, or for confidential VMs where cross-socket access is encrypted over the link, the UPI headroom is what keeps remote access from becoming the bottleneck the extra memory bandwidth was supposed to remove. For the deeper layout argument, the v5 hardware and Ceph editorial covers how these same nodes shape distributed-storage behavior.
The drive interface did not widen, and that tells you where the bottleneck went
The most useful thing to say about v5 bandwidth is where it did not change. The Micron 7500 MAX, the standard v5 data drive, is a PCIe Gen4 device by interface (per the Micron 7500 Tech Prod Spec). A 6.4 TB unit reads sequentially around 7 GB/s and sustains roughly 1.1 million random read IOPS with a six-nines QoS floor under 1 ms. Those are strong numbers, and they did not move with the generation, because the drive interface stayed at Gen4 while the network, memory, and interconnect all widened around it.
That is the point, not a disappointment. When every link upstream of the disk gets wider and the disk interface holds, the place a workload bottlenecks shifts off the network or the memory bus and back toward the media and the access pattern. For an architect, that is clarifying. It means v5’s gains land on the traffic that moves between drives, sockets, and nodes, the replication and recovery and east-west paths, rather than on single-drive throughput. Size the cluster for the data movement, not for the spec of one SSD. The Large v5 and XL v5 detail pages list the per-tier specs that anchor that sizing.
The bandwidth you would be billed for elsewhere rides the private fabric
There is a billing consequence worth stating plainly, because it changes how freely you can use the wider pipe. The 40 Gbps fabric carries internal traffic, and internal traffic between servers on OpenMetal is not metered. Replication, recovery, backfill, live migration, and Kubernetes east-west traffic all live on that fabric. On a per-gigabyte hyperscaler egress meter, a large recovery event or a cross-node rebalance would meter every object it moved. Here that movement is internal and unbilled, so the wider pipe is free to use for exactly the bursty internal work it was widened for.
Public egress, the traffic that leaves your VLANs for the internet, is billed after the included base on a fair model rather than a per-gigabyte meter. OpenMetal bills it on a 95th-percentile basis, so short bursts are smoothed across the sample window instead of charged by the gigabyte. The engineering point is that the traffic the v5 bandwidth upgrades unlock is overwhelmingly internal, and internal traffic does not meter.
What the wider pipe adds up to
v5 is a bandwidth generation even more so than a clock generation. The fabric doubled, memory bandwidth rose by roughly a quarter to a third per socket, the lane budget and interconnect grew, and the drive interface deliberately did not. For a hyper-converged core, that combination shortens recovery and rebalance windows, raises how densely you can pack a node before east-west traffic saturates, and keeps the cost of that data movement off the bill because it stays on the unmetered private fabric. The two things to hold onto when you size it: fill the memory channels on Medium and Large v5 if the workload is memory-bound, and remember that with every upstream link wider, the bottleneck now sits closer to the media and the access pattern than to the network.
Explore the v5 hardware catalog
The way to know what the wider fabric does for your workload is to put your own replication and recovery patterns on it. Browse the v5 line in the hardware catalog to compare the per-tier bandwidth, memory channel population, and lane budgets against what your architecture actually moves between nodes.