OpenMetal Enterprise Storage Tier Offerings and Architecture

High Performance NVMe (Direct Attached Storage)

This tier utilizes direct-attached NVMe drives installed in the compute or hyper-converged nodes for maximum I/O performance. VMs can use these NVMe disks as local block devices (often for ephemeral storage or caching), bypassing network storage layers for the lowest possible latency. In this configuration, a VM’s data resides on the host’s NVMe drive, delivering extremely high IOPS and throughput .

Use Cases

Workloads that demand very fast storage access and can tolerate or mitigate limited hardware redundancy. Examples include high-performance databases or analytics engines that replicate data at the application level. For instance, a distributed database cluster (e.g. MySQL Galera) can place its data on local NVMe devices to benefit from high IOPS, while relying on its own replication across nodes to maintain data integrity .

Availability Characteristics

Direct-attached NVMe storage is not inherently highly available at the cluster level – data is not automatically replicated to other servers. If the host fails, any VM data on its local NVMe could be lost (unless the application has its own replication). To provide partial fault tolerance, operators often configure software RAID (e.g. RAID 1 mirroring) across two NVMe drives in the same server . This protects against an NVMe device failure by maintaining a redundant copy on a second drive, with a minimal write performance penalty and improved read performance. However, even with RAID, the data is confined to that single server (“partially HA”), so a complete node failure would still require restoration of the affected VMs from backups or application-level replication.

Where does it work best

Many performance-sensitive applications handle high availability in software or can tolerate ephemeral storage. By using the NVMe locally, these workloads achieve maximum performance, and they accept the trade-off of handling HA themselves . In practice, this is often acceptable for ephemeral VM disks, cache data, or systems designed with redundancy (e.g. a caching layer, or stateless services). Additionally, direct NVMe provides a simpler storage path (no network overhead), which can be crucial for ultra-low latency needs. Administrators should note that using local storage for VMs means features like live migration are not available for those instances (since the storage is tied to one host) . Even so, the combination of raw speed and the option of RAID for disk-level protection makes this tier attractive for specific high-performance scenarios.

Browse Bare Metal Catalog

Questions? Schedule a meeting or start a chat.

Mid to High HA Performance Storage (Network Attached Storage)

This tier is built on network-attached storage using a Ceph cluster, providing high availability (HA) through data replication while still offering strong performance. OpenMetal deploys Ceph (an open-source distributed storage system) on dedicated storage servers to furnish block storage (and also file/object, if needed) to the cloud. In a typical setup, each storage node contributes drives to Ceph, and OpenMetal’s storage server hardware supports both NVMe SSDs and spinning HDDs. By leveraging these, OpenMetal can create separate Ceph storage pools for different performance characteristics – for example, an HA-NVMe pool (using NVMe SSD OSDs) and an HA-Spinner pool (using HDD OSDs) . This separation follows Ceph best practices (not mixing media types in one pool) and effectively provides tiered performance within the HA storage layer.

Remote NVMe-Based Ceph Cluster

For high IOPS needs with built-in redundancy, OpenMetal offers Ceph pools backed by NVMe drives. In this configuration, data written to a volume is replicated across multiple NVMe OSDs on different servers (3 replicas for HA). This yields much higher I/O than spinning disks, while Ceph ensures that even if one node or drive fails, the data remains available from other replicas . The result is a high-performance, highly-available storage service – ideal for running virtual machine volumes that require fast throughput (such as transactional databases or intensive VM workloads) without sacrificing fault tolerance. Because it’s network-based, this storage allows live migration of VMs between hypervisors (the shared storage means a VM can move hosts with minimal downtime) , and the cloud can tolerate a server outage with no data loss or VM downtime beyond failover delays.

Hybrid NVMe+HDD Architecture

OpenMetal’s storage servers (see comparison table below) contain both fast NVMe SSDs and high-capacity HDDs. Ceph capitalizes on this by letting cloud operators use high-speed NVMe for “hot” data and metadata, and HDD for “cold” or less performance-sensitive data. In practice, an OpenMetal private cloud might have two volume types exposed in OpenStack: one backed by an NVMe pool (for latency-sensitive volumes) and one by an HDD pool (for bulk storage at lower cost). Cloud users can choose the storage type when provisioning volumes or launching instances, aligning performance vs. cost to their needs. Both pools are fully HA, as Ceph replicates data across at least three drives/hosts by default. Ceph’s design also allows scaling: adding more storage nodes increases capacity and performance linearly, and Ceph will rebalance data across new drives automatically.

OpenMetal Implementation

In OpenMetal’s architecture, the initial private cloud deployment (the “Cloud Core”) comes with a Ceph cluster configured for block storage. Each server (even those primarily for compute) has at least one drive allocated to Ceph by default, creating a redundant storage pool available to OpenStack Cinder. The NVMe and HDD pools are presented through OpenStack as different storage tiers (often as separate volume types or performance tiers). For example, an operator might designate an “SSD Volume” type mapped to the NVMe-backed Ceph pool and a “Standard Volume” type to the HDD pool. Under the hood, Ceph ensures data durability; with replication (e.g. 3 copies), the storage system continues serving data even if one server or disk goes down . This mid-to-high performance tier thus covers a wide spectrum: mid-performance if using HDD-based volumes (due to network latency and disk speed) and high-performance if using NVMe-based volumes – all with the safety net of HA. It’s a balanced choice for general-purpose cloud workloads where some level of performance is needed along with strong reliability.

Browse Bare Metal Catalog

Questions? Schedule a meeting or start a chat.

Low Cost Object Storage (Network Attached Storage)

For massive storage needs and cost efficiency, OpenMetal provides a low-cost object storage tier powered by a large Ceph cluster predominantly using spinning disks. This is ideal for archiving, static content, big data lakes, backups, and any use case where capacity and durability are prioritized over raw IOPS.

Let’s for example, take a dedicated object storage deployment consists of 20 Large v4 storage servers populated with high-capacity HDDs . The Ceph cluster on these nodes is configured with an erasure coding scheme of 17 data to 3 parity (17+3), meaning data is split into 17 chunks plus 3 coding chunks for redundancy. This configuration can tolerate the failure of up to 3 drives or nodes without data loss, while using storage space much more efficiently than full replication (the overhead is only ~17.6% extra, versus 200% overhead for triple replication). In real terms, the 20-node, HDD-based cluster provides on the order of 2.7 PiB of usable storage (~3.2 PiB raw) at 85% utilization.

Architecture Details

If we break down the architecture for the above example, each Storage Large V4 server in this object storage cluster is equipped with a large number of spinner disks (HDDs) for capacity, and also a set of NVMe SSDs as accelerator devices. For instance, a typical node may have 12× 18 TB HDDs for main object storage, complemented by 4× 4 TB NVMe drives that act as BlueStore DB/WAL devices or caching tiers to boost performance for Ceph operations . The NVMe accelerators handle metadata and small writes efficiently, allowing the HDD-based OSDs to focus on bulk data. This mixed approach leverages Ceph’s ability to use fast media to improve the performance of large slow media pools. The object storage is accessed via Ceph’s RADOS Gateway (RGW), which presents an S3-compatible API endpoint for users. From the OpenStack side, this object store can be integrated as well (accessible as a service in the cloud for storing images, backups, etc., independent of block storage).

Why Spinners + Erasure Coding

By using predominantly spinning disks with erasure coding, OpenMetal achieves a very low effective cost per terabyte of storage while still providing fault tolerance and good throughput for sequential reads/writes. Erasure coding spreads data across many drives, so large read/write operations can be parallelized across 17 disks, achieving high aggregate throughput – suitable for object storage workloads. The trade-off is slightly higher latency for small updates (due to compute overhead of coding and the need to read/update multiple fragments), but object storage use cases (large files, infrequent updates) are well-suited to this. The inclusion of some NVMe in each node mitigates this latency by speeding up internal Ceph metadata and write-ahead logging. In summary, this tier is optimized for capacity and cost: it provides huge scalable storage (on the order of petabytes) at an affordable price, with the Ceph cluster ensuring no single point of failure. OpenMetal’s support for both HDD and NVMe in the storage nodes means even a “cold storage” cluster can benefit from a bit of flash to accelerate key operations.

Comparison of OpenMetal Storage Tiers

Benefits of OpenMetal Tiered Storage Options

Having multiple storage tiers in OpenMetal’s cloud environment yields several key benefits for operators and users:

• Cost Optimization: Each tier offers the best price for the performance it delivers. By storing data on the least expensive appropriate tier, organizations can significantly reduce costs. For example, archival data can reside on the erasure-coded HDD object store (lowest $/GB), while only mission-critical hot data uses premium NVMe storage. This alignment prevents overspending on high-end storage for workloads that don’t require it . In essence, you pay for performance only where needed and use economical capacity storage for the rest.
• Performance Alignment: Workloads can be matched to the storage that best meets their performance profile. Applications requiring microsecond latency and huge IOPS (like real-time analytics or high-frequency trading apps) can use direct NVMe or NVMe-backed volumes, whereas workloads needing decent throughput but not low latency (like archival, media streaming, or backups) can use HDD-backed volumes or object storage. By tiering storage, OpenMetal ensures that high-performance resources (NVMe, high-end CPUs) are available for the workloads that truly need them, while other workloads are not penalized – they get adequate performance from standard tiers without contending with the most demanding apps.
• High Availability & Resiliency: Tiered storage lets users choose their desired level of redundancy. If a workload requires built-in HA, one can use the Ceph-backed network storage (with replication or erasure coding providing data protection). If a workload instead handles HA at the software level (or can tolerate data loss), it can use non-replicated local NVMe for speed. This flexibility means the overall system can be architected for reliability in a cost-effective way – e.g., replicate critical data on Ceph (which tolerates disk/server failures transparently) , while using local disks for scratch data or where the application does its own replication. Having Ceph’s highly available storage alongside ephemeral local storage gives cloud operators a spectrum of fault-tolerance options for different scenarios.
• Scalability and Flexibility: A multi-tier storage architecture is inherently more flexible. As demands grow, each tier can be scaled independently. Need more fast block storage? Add another NVMe-based storage node or expand the NVMe pool. Need more capacity? Add more HDD-based nodes to the object storage cluster. This modular growth ensures that the infrastructure can evolve with workloads. Furthermore, OpenMetal’s integration of these tiers under a single OpenStack environment means that from the user’s perspective, it’s seamless – they simply select a volume type or storage backend, and the cloud handles where the data lives. It simplifies management while still providing granular control to tune cost/performance. In short, tiered storage provides a balanced approach to cloud design, enabling fault tolerance where required, speed where it’s most beneficial, and cost-savings everywhere else.

Browse Bare Metal Catalog

Questions? Schedule a meeting or start a chat.