Ceph Usable Storage on a 3-Node Hosted Private Cloud XL v5

Q: What is the usable storage capacity of a 3-node Hosted Private Cloud XL v5?

A 3-node Hosted Private Cloud XL v5 has 76.8 TB of raw NVMe across 12 Ceph OSDs and delivers approximately 25.6 TB of usable Ceph-replicated storage at the default 3x replication factor.

Schedule a Consultation

The math: each XL v5 node carries four 6.4 TB Micron 7500 MAX U.3 NVMe drives, each running as a single Ceph OSD. Across three nodes that is 12 OSDs at 6.4 TB each, or 76.8 TB raw. At the cluster’s default 3x replicated pool (each object written to three OSDs across three failure domains), the usable capacity lands near one-third of raw — approximately 25.6 TB once Ceph overhead, reservation, and rebalance headroom are accounted for. The cluster survives the loss of a full XL v5 node without data loss because the failure domain is node-level.

For workloads where 3x replication overhead is acceptable, Cinder block volumes and Nova ephemeral VM disks default to the replicated pool — that latency profile is what most general-purpose VM workloads expect. For workloads where capacity is more important than write latency, erasure coding pools can be configured per-pool: a common pattern is 4+2 EC on a pool dedicated to RADOS Gateway (S3-compatible object storage) or archival data, which trades some write latency for usable capacity closer to 67% of raw.

The all-NVMe OSD layout is meaningful. There is no HDD bulk tier and no separate SSD journal tier — every Ceph operation is end-to-end NVMe on PCIe Gen4 hardware, with sub-1ms 6-nines QoS on the underlying drives. The cluster behaves more like a SAN array on flash than a typical Ceph deployment from the early 2010s. Customers needing more raw capacity than 76.8 TB scale the cluster horizontally by adding XL v5 nodes (each adds 25.6 TB raw / ~8.5 TB usable) or move to storage-optimized SKUs with HDD bulk tiers for cold data.

—-