OpenMetal AI Deployment Guides · Training · The Buyer’s Guide

When to fine-tune, what to fine-tune, and what it takes

Fine-tuning is worth it when you need to change how a model behaves, not just what it knows. This guide walks the real projects companies run, the method each one calls for, and the OpenMetal box it belongs on. It also helps you decide whether you should fine-tune at all, or reach for retrieval instead.

Why run fine-tuning on OpenMetal

The case for owning the box for training

Your training data stays put

Fine-tuning data is often your most sensitive material: customer records, source code, internal documents. On dedicated hardware it never leaves infrastructure you control.

Fixed cost for sustained runs

Fine-tuning runs for hours or days. A flat monthly box makes that predictable, where a metered per-hour service turns a long run into an unpredictable bill.

Iterate without the meter running

Fine-tuning is rarely one run; it is many small experiments. Owning the box means you can iterate freely without watching a usage clock.

Train and serve in one place

Fine-tune on a GPU box, then serve the result on the right box (including the CPU box for small distilled models), all on infrastructure you already have.

At a glance

Project to method to box

Project	Typical method	Box	RAG instead?
Domain adaptation	QLoRA (full for deep shift)	H200 / RP6000	Sometimes
Instruction / format tuning	LoRA / QLoRA	RP6000 / H200	No
Brand voice & tone	QLoRA	RP6000	No
Add / improve a language	LoRA or full	H200 / larger	No
Tool-calling reliability	LoRA	RP6000 / H200	No
Distillation to a small model	Full / LoRA on student	H200 / RP6000	No
Classifier / extractor	LoRA (small model)	RP6000	No
Continued pre-training	Full, large corpus	Larger deployment	Usually
Safety / guardrail tuning	LoRA	RP6000 / H200	No

Box column is a starting recommendation. Final choice depends on the exact model, dataset size, and how fast you want to iterate, which is what a sizing conversation resolves. See the training fit guide for the model-by-method matrix.

Should you fine-tune?

Fine-tune to change behavior, retrieve to add knowledge

The most common mistake is fine-tuning when retrieval would have been simpler and cheaper. A quick test before any project:

Use RAG, no training, when the need is “answer from our documents” or “keep up with changing facts.” Retrieval handles knowledge, updates instantly, and cites sources.
Fine-tune when the need is a change in behavior: a consistent output format, a domain style or tone, a specific language, reliable tool-calling, or a small fast model that imitates a big one.
Do both when you want a model that behaves a certain way and stays current: fine-tune the behavior, use RAG for the facts.

The inference use-case guide covers the RAG side in depth. Everything below is for when fine-tuning is the right call.

The projects

Nine fine-tuning projects, mapped to method and box

Each card gives the project, an example, the method it usually calls for, and the box. “Method” is the lever from the fit guide: QLoRA (cheapest, single card), LoRA (mid), or full fine-tuning (heaviest).

Domain adaptation

QLoRA / LoRAGPU

Teach a general model the vocabulary, conventions, and reasoning of a specialized field so it stops sounding like a generalist.

Example: adapt a 70B model to legal drafting so it uses the right clause structures and terms of art.

MethodQLoRA usually suffices; full fine-tune only for a deep domain shift.

BoxH200 for 70B-class QLoRA; RP6000 for smaller.

RAG instead?If it is really “look up our documents,” use RAG. Fine-tune for the style and reasoning.

Instruction & output-format tuning

LoRA / QLoRAGPU

Make the model reliably follow your instructions and emit a fixed structure (JSON schema, report template, ticket format) every time, without prompt gymnastics.

Example: a model that always returns valid JSON with your exact fields for a downstream system.

MethodLoRA or QLoRA on a mid-size model; format tuning is data-light.

BoxRP6000 or H200; a 14B to 32B model is usually enough.

RAG instead?No; this is behavior, not knowledge.

Brand voice & tone

QLoRAGPU

Make outputs sound like your company: the register, phrasing, and do-not-say rules of your style guide, consistently.

Example: a support-reply model that matches your brand voice across thousands of messages.

MethodQLoRA on a small-to-mid model; a few hundred good examples go far.

BoxRP6000 comfortably.

RAG instead?No; tone is learned, not retrieved.

Add or improve a language

LoRA / fullGPU

Raise quality in a language the base model handles weakly, or add a low-resource language your customers use.

Example: strengthen a model’s performance in a regional language for local support.

MethodLoRA for improvement; a deep language addition can justify full fine-tune or continued pre-training.

BoxH200 for larger runs; a full fine-tune of a big model is scoped with us.

RAG instead?No; language ability lives in the weights.

Tool-calling & function-calling reliability

LoRAGPU

Make an agent reliably choose the right tool and produce well-formed calls, which is often the weak link in agent workflows.

Example: tune a model to call your internal APIs with correct arguments nearly every time.

MethodLoRA on examples of correct tool use.

BoxRP6000 or H200 depending on model size.

RAG instead?No; this is behavior. Pairs well with the agent use case in the inference guide.

Distillation: shrink a big model into a small one

LoRA / fullGPU

Train a small, cheap-to-serve model to imitate a larger one on your task, so day-to-day inference runs on lighter hardware.

Example: distill a 70B model’s behavior into an 8B student that serves interactively on a smaller box.

MethodFull fine-tune or LoRA on the small student, using the big model’s outputs.

BoxH200 or RP6000; the student is small, so it fits easily.

PayoffCuts ongoing inference cost; a training investment that pays back in serving.

Purpose-built classifier or extractor

LoRAGPU

Turn a general model into a fast, accurate specialist for one job: routing, tagging, sentiment, or pulling fields from text.

Example: fine-tune a small model to extract structured fields from contracts far more accurately than prompting alone.

MethodLoRA on a small model, from labeled examples.

BoxRP6000; small models train quickly.

RAG instead?No, though the CPU box then serves the result cheaply at inference.

Continued pre-training on a proprietary corpus

full / larger deploymentGPU

Keep training a base model on a large body of your own text so the domain becomes native, not bolted on. The heaviest project here.

Example: continue training on years of internal engineering documents to build a deeply in-domain base.

MethodFull training over a large corpus; not a quick adapter run.

BoxA larger deployment scoped with OpenMetal directly.

RAG instead?Usually yes for pure knowledge; reserve this for genuine deep-domain needs.

Safety & guardrail tuning

LoRAGPU

Teach the model your refusal rules, escalation behavior, and boundaries, so it handles sensitive requests the way your policy requires.

Example: tune a support model to decline out-of-scope requests and route them correctly.

MethodLoRA on policy examples; often paired with a small guardrail classifier.

BoxRP6000 or H200.

RAG instead?No; behavior, not knowledge.

Let us scope your fine-tuning project

Tell us what you want the model to do differently and roughly how much data you have. We will suggest a method, a base model, and a box. Proof-of-concept environments are available for testing before you commit.

Pricing for GPU configurations is provided on request.

Talk to an engineer See the fit matrix

Reference

Glossary and models reference

Every term used across these guides, plus the full table of models with owner and license, lives on the AI Deployment Guides hub.

Companion guides: How fine-tuning works (the concepts) and What you can fine-tune (model-by-method fit). For running finished models, see the inference use-case guide.

Project-to-box recommendations are starting points for a scoping conversation, not guarantees; final sizing depends on your model, dataset, and iteration speed. Model availability and licensing should be confirmed with OpenMetal.