DeepSeek-R1’s 671B MoE fits one eight-GPU HGX B300 node at FP8, with room for KV cache. See the fit math and why the node beats a multi-server build.
Tag: Mixture of Experts
DeepSeek-R1 is a 671B MoE: its FP8 weights span about six discrete H200 cards. See the fit math, why MoE memory follows total params, and how to serve it.
A first-party FP4 checkpoint moves GPU selection from memory capacity to native tensor-core format support, and inverts the usual verdict. Based on the Inkling-Small.