What you can fine-tune on each OpenMetal box
A reference for sizing fine-tuning runs on the H200 and RP6000 (the CPU box is for data prep, not gradients). The footprint is set by your method: full fine-tuning holds weights plus gradients plus optimizer state, LoRA holds the frozen model plus small adapters, and QLoRA holds a 4-bit frozen model plus adapters. If those terms need unpacking, start with the explainer.
The hardware
Two training boxes, and one for the prep
Fine-tuning is compute-heavy, so it runs on the GPU boxes. The larger the card’s memory, the more you can fit on one card. New to the method terms? See the training explainer. Owner and license for each model are in the models reference on the hub.
H200
The strongest single-box option for fine-tuning. QLoRA on models up to the 100B class, LoRA on mid-size models, and full fine-tuning of smaller models on one card.
Full hardware specs →RP6000
Solid for fine-tuning at a tighter ceiling. QLoRA up to the 70B class, LoRA on smaller-to-mid models, full fine-tuning of small models.
Full hardware specs →XL v5 CPU
Not a training box: training is compute-bound. Its role is preparing and cleaning datasets, tokenizing, and orchestrating runs.
Full hardware specs →
The fit matrix
Model × method × box
Single card throughout. Cells assume gradient checkpointing and sensible batch and sequence-length settings; larger batches or longer sequences push a job up a tier. These are reasoned starting points, not measured limits.
| Model | Full fine-tune | LoRA (16-bit base) | QLoRA (4-bit base) |
|---|---|---|---|
| ~8B classe.g. Llama 3.1 8B | 1 cardH200; RP6000 tight | 1 cardboth | 1 cardboth, easily |
| Phi-414.7B | 1 card, offloadH200; slower | 1 cardboth | 1 cardboth |
| Qwen3.6-27B27B | Beyond 1 card | 1 cardH200; RP6000 tight | 1 cardboth |
| Gemma 2 27B27.2B | Beyond 1 card | 1 cardH200; RP6000 tight | 1 cardboth |
| Qwen2.5-Coder 32B32.5B | Beyond 1 card | 1 cardH200; RP6000 tight | 1 cardboth |
| Qwen3.6-35B-A3B35B / ~3B active, MoE | Beyond 1 card | 1 cardH200 only | 1 cardboth |
| Llama 3.3 70B / Qwen2.5 72B70 to 73B | Beyond 1 card | 1 card, tightH200 only; RP6000 no | 1 cardboth |
| 100B+ classCommand R+, Mistral Large 2 (non-commercial license) | Beyond 1 card | Beyond 1 card | 1 cardH200; RP6000 tight |
The clear pattern: QLoRA keeps almost everything on one card, LoRA covers mid-size on one card, and full fine-tuning is single-card only for smaller models. Anything marked Beyond 1 card is a larger deployment we scope with you directly, rather than something to plan from this table.
A few things the matrix assumes
- Mixture-of-Experts models (Mixtral, DeepSeek) can be fine-tuned by the same methods, but their footprint follows the full stored size (all experts), so treat them by total size, not active size. Fine-tuning MoE well is more specialized; talk to us before planning one.
- These guides size single-card runs. A run above a single card’s ceiling is a larger deployment scoped with OpenMetal directly; the table covers what fits one card.
- Offload trades speed for fit. A job slightly too big for a card can spill optimizer state to system memory, which lets it run on one card but slows it down. Useful for a one-off, not for a fast iteration loop.
The short version
For most fine-tuning, QLoRA on a single card is the right first move: it fits models up to the 70B class on the RP6000 and to the 100B class on the H200, trains quickly, and reaches most of full fine-tuning’s quality. Move to LoRA or full fine-tuning only when a specific quality goal requires it, and talk to us for anything beyond a single card.
Which projects call for which method? See the training use-case guide for real fine-tuning projects mapped to method and box.
Let us size your fine-tuning run
Tell us the model, your dataset size, and what you want it to learn, and we will map it to a method and a box. Proof-of-concept environments are available for testing before you commit.
Pricing for GPU configurations is provided on request.
Talk to an engineer See training use casesReference
Glossary and models reference
Every term used across these guides, plus the full table of models with owner and license, lives on the AI Deployment Guides hub.
Companion guides: How fine-tuning works (the concepts) and training use cases (projects mapped to method and box). For running finished models, see the inference fit guide.
Fit calls are reasoned from method, model size, and card memory, not measured. Actual limits depend on batch size, sequence length, optimizer, and offload settings; OpenMetal can size and benchmark a specific run on request. Model availability and licensing should be confirmed with OpenMetal.