Self-Hosting a Closed Model: What’s Actually Possible on OpenMetal

1. What can and cannot be self-hosted

Start with the fact that reframes everything: the frontier models are closed-weight. OpenAI’s flagship GPT models are API-only, Google’s Gemini is a closed API product, and Anthropic ships no open weights at all. There is no checkpoint to download, quantize, and serve. Inference happens on the provider’s cloud, or on a provider-operated partner cloud, and never on a machine you control.

This is worth stating precisely because the same vendors also ship open-weight families, and the two are easy to confuse. OpenAI released gpt-oss (Apache 2.0) in 2025, Google publishes Gemma, and Meta publishes Llama. Those you can download and run on your own GPUs. But they are not the flagship model. If your requirement is specifically Opus 5, GPT, or Gemini, you are using an API, full stop.

So a request to “self-host a frontier model” really means hosting the system around it, because the model itself is not something you can host. The table below is the honest inventory of where each layer can run.

The single unavoidable movement is the last two rows: the prompt goes out and the completion comes back. Everything above them can be yours.

2. The deployment route archetypes

Every provider offers some subset of four routes for keeping more of the system inside your boundary. None of them runs the model on your hardware.

• API direct. Your application, running on your infrastructure, calls the provider’s API with your key. The simplest route, and the one all the others build on.
• Self-hosted sandbox. The provider runs the model and the agent loop; the agent’s tool execution runs in a container you control. Anthropic shipped exactly this for Claude Managed Agents in mid-2026 (public beta): orchestration, context management, and error recovery stay on Anthropic-managed infrastructure, while tool execution and private-network access move to the customer side. The practical effect is that code, files, and internal systems the agent touches stay inside your perimeter, and the execution layer becomes eligible for stricter data-handling terms. 
• Cloud-partner route. The model is served through a hyperscaler you may already have a contract and data agreement with (Amazon Bedrock, Google Vertex AI, Microsoft Foundry, Azure OpenAI). This does not put the model on your metal, but it can change which legal entity holds the data and where the endpoint lives.
• Self-hosted UI. An open-source chat front-end on your servers, connected to the API with your key. Your users, prompt templates, and conversation history stay in your database instead of a vendor’s.

The pattern across all four: the further you go, the more of the surrounding system sits inside your boundary, and the more control you have over the one thing that still crosses. The model never moves. For the exact routes and data-handling terms each provider offers, see the provider-specific guides: self-hosting your Claude stack, your GPT stack, and your Gemini stack.

3. What you host on OpenMetal

The layers you can own are ordinary compute: an application tier, a database, a proxy. They do not need a GPU, and they benefit from being on single-tenant infrastructure with a private network and predictable cost. That is what OpenMetal provides.

Concretely, a Hosted Private Cloud or Bare Metal deployment gives you a single-tenant environment where the chat UI, the agent sandbox, the retrieval corpus and its vector database, your logs, and the egress proxy all run on hardware only you use. Traffic between those components is east-west private-network traffic, carried on per-customer private networks and included rather than metered, so your retrieval and logging layers can be as chatty as they need to be without a transfer bill. Public egress ships with a generous included allotment, which matters because the model calls themselves are the only traffic that needs to leave. To be clear about hardware: this stack needs no GPU. It is web application, database, and proxy compute. A GPU enters the picture only if you also choose to self-host an open-weight model yourself, which is the hybrid pattern in the next section and a separate deployment; the closed model always runs on the provider’s API, never on your silicon.

The guardrail and egress-proxy tier is the part worth designing carefully, because it is where sovereignty is actually enforced. Sitting at the single north-bound egress point, on hardware you control, it inspects every outbound prompt: redacting or tokenizing sensitive fields, applying policy on what may be sent, and logging exactly what crossed. Because it is your infrastructure and not a vendor setting, the rule about what leaves is yours to write and yours to audit.

What to run it on

Because there is no GPU in this picture, the sizing question is about cores, memory, and isolation, not accelerators. Two sensible starting points:

Either way the model still runs on the provider’s API; the box you choose is sized for retrieval, logging, and the egress proxy, never for inference.

4. The honest ceiling

This is the section that keeps the rest credible. State it plainly:

• The model never runs locally. Closed frontier models are closed-weight. There is nothing to download and nothing to load on a GPU. If a vendor or a blog post implies a flagship closed model can be served from your own hardware, it is wrong, or it means the open-weight cousin, not the flagship.
• Prompt and completion payloads still transit to the provider. You can govern the shape of what leaves, but you cannot make the model call local. Sovereignty here is boundary control, not zero egress.
• A provider agreement is separate from your infrastructure’s compliance. OpenMetal is HIPAA compliant at the organizational level, but that posture covers your infrastructure, not the API call. If you send regulated data to a model, you need a Business Associate Agreement (or the equivalent enterprise data-handling terms) with the model provider, on the right plan. The two are additive, never a substitute for each other. 
• Air-gapped and classified workloads exclude closed models entirely. If nothing may leave the network, no API route qualifies, no matter how the surrounding stack is built. Those workloads belong on an open-weight model running on your own metal, which is the full-sovereignty tier and a different deployment.

5. The sovereignty spectrum, and the bridge

Everything above lands on a three-tier spectrum:

• Convenience. The model, data, and application all sit on the provider’s managed cloud. Least in your boundary, least to operate, and the tier OpenMetal displaces.
• Data-plane sovereignty. The application and data plane run on your single-tenant OpenMetal environment; the model is called through a governed API egress. Prompts still leave, but on your terms. This is the tier this series is about.
• Full sovereignty. An open-weight model runs on your own hardware, air-gapped if you need it. Nothing leaves. This is not a closed-model deployment at all; it is the subject of our Model Deployment Fit series, where open-weight models like Llama, gpt-oss, and Gemma run on OpenMetal GPU servers.

A quick way to place yourself: if any data must never leave your network, you are in full sovereignty, which means an open-weight model on your own metal. If you need a frontier model’s quality but must control what it sees, you are in data-plane sovereignty, where most regulated teams land and where the rest of this series lives.
The two right-hand tiers are not mutually exclusive. A hybrid design routes the bulk of traffic, and anything sensitive, to an open-weight model on your own OpenMetal GPUs behind the same egress proxy, and escalates only the hardest prompts to a closed model’s API. That shrinks what leaves your boundary and cuts token spend at the same time. It is the practical bridge between the data-plane and full-sovereignty tiers.

6. Why OpenMetal for the data plane

The reason to run this stack on OpenMetal rather than a hyperscaler is that the value of the data-plane tier depends on the boundary being real. Single-tenancy means your vector store and your logs sit on hardware no other tenant shares. Private per-customer networking means the chatter between your app, your retrieval layer, and your proxy never touches the public internet. Included egress and fixed monthly pricing mean the architecture does not punish you for keeping data close, the way metered cross-zone and egress fees do elsewhere.

There is a strategic payoff too. Because the model is just a call behind your proxy, your data plane is provider-neutral. You can point it at Claude today, add GPT or Gemini tomorrow, fail over between them, or A/B one against another, without moving a byte of your corpus, your logs, or your guardrails. You own the substrate; the model is a swappable backend. That is the opposite of the lock-in that comes from building your whole stack inside one vendor’s managed platform.