Prefill is compute-bound, decode is memory-bandwidth-bound. Why splitting inference into two purpose-fit GPU pools beats one uniform fleet.
Tag: Hardware Editorial
After weights load, the HBM left over is your KV-cache budget. Why the H200’s 141GB buys more context and concurrency than a 94GB H100.
Map MongoDB, Redis, Kafka, ClickHouse, and Kubernetes workers to OpenMetal SKUs by the resource each role saturates, then size the failure domain.
An ordered Day-2 playbook for a single-tenant H200: full root and IPMI, owning the CUDA stack, boot-data isolation, and a node-bounded blast radius.
The v5 generation can be told as a cores-and-clocks story, but a significant change is bandwidth: the private fabric doubled to 40 Gbps, memory moved to DDR5-6400, and the lane budget grew to 88 PCIe 5.0 lanes.
All-NVMe OSDs, an isolated boot pool, a clean lane budget, and identical nodes: how OpenMetal’s v5 hardware makes Ceph behave predictably instead of needing tuning.