GPU Memory on the OpenMetal RP6000

Q: How much GPU memory does the OpenMetal RP6000 have?

Each OpenMetal RP6000 GPU carries 96GB of GDDR7 memory, and a server can hold one or two cards for up to 192GB of GPU memory.

Explore GPU servers

The 96GB of GDDR7 runs at 1.79 TB/s on a 512-bit bus. For training and fine-tuning that capacity holds sizeable models and batches on a single card; for inference it allows larger KV caches and higher batch sizes. It is roughly double the memory of common Ada-generation inference GPUs such as the L40S (48GB).

GPU-memory pooling is available between the two GPUs within a single server. Across multiple servers in a cluster, GPUs communicate over the private 40 Gbps network using data and pipeline parallelism rather than a shared GPU-memory fabric, so workloads that need tightly coupled GPU memory should be sized to the per-server card count.

Separately, each server ships with 1TB of DDR5-6400 host memory (upgradeable to 2TB) that stages datasets, vector indexes, and embeddings to keep the GPU fed. When a workload needs more than 96GB per card, OpenMetal’s H200 (141GB HBM3e) is the larger-memory option.