Self-Hosting Your GPT Stack on OpenMetal

1. What runs where

OpenAI’s flagship is closed-weight. It runs on OpenAI’s infrastructure, or on Azure OpenAI, and never on a machine you control. (OpenAI’s open-weight cousins, gpt-oss-120b and gpt-oss-20b, you can run yourself, but they are not the flagship.) That fixes one row of the stack on the provider’s side; the rest can be yours.

The only data that must leave is the prompt and its completion. Your egress proxy decides what each prompt actually carries.

2. Deployment options for GPT

OpenAI offers three practical routes for keeping more of the system inside your boundary. None puts the model on your hardware.

• API direct. Your application on OpenMetal calls the OpenAI API with your key. Under the commercial terms, OpenAI does not train on your API data. Zero data retention, which stops prompts and completions from being stored after the response, is available but is not a default: it is an enterprise feature you must request and be approved for. [VERIFY] current ZDR eligibility for your account.
• Azure OpenAI (cloud-partner route). The route that most changes your data posture. Microsoft commits that customer prompts, completions, and embeddings are not used to train foundation models without your permission, and offers deployment types with distinct residency semantics: Standard processes within a geography you specify, DataZone processes within a named data zone with data at rest in your designated geography, and Global processes wherever the model is deployed with data at rest pinned to your geography. If you already hold an Azure data agreement, this can change which entity processes the data and where. [VERIFY] current deployment types and regions.
• Self-hosted UI. An open-source chat front-end on your OpenMetal servers, connected to the API. Your users, prompt templates, and conversation history live in your database.

3. What you host on OpenMetal

Every layer you own here is ordinary compute. It does not need a GPU, and it benefits from single-tenant hardware, a private network, and predictable cost.

On a Hosted Private Cloud or Bare Metal deployment, the chat UI, your agent tooling, the retrieval corpus and its vector database, your logs, and the egress proxy all run on hardware only you use. Traffic between them is east-west private-network traffic on per-customer private networks, included rather than metered, so your retrieval and logging layers can be as busy as they need without a transfer bill. Public egress ships with a generous included allotment, which is all the GPT calls need.

The guardrail and egress-proxy tier is the part to design deliberately. Sitting at the single north-bound egress point on hardware you control, it inspects every outbound prompt: redacting or tokenizing fields, applying policy on what may be sent, and recording exactly what crossed. Because it is your infrastructure rather than a provider setting, the rule for what leaves is yours to write and yours to audit.

4. The honest ceiling

State the limits plainly, because they are what make the rest trustworthy.

• GPT never runs locally. The flagship GPT is closed-weight. There is nothing to download and nothing to load on an OpenMetal GPU. If you need weights you can host, that is gpt-oss, the open-weight cousin, not the flagship.
• Prompts and completions still transit to OpenAI or Azure OpenAI. The no-training commitment and an optional zero-data-retention configuration govern how that data is handled, but they do not keep it inside your network. Sovereignty here is boundary control, not zero egress.
• A GPT BAA is available, but conditional, and separate from OpenMetal’s HIPAA. OpenAI signs a Business Associate Agreement for eligible enterprise customers, and once the org is provisioned it can process PHI on BAA-eligible endpoints, but surfaces such as Web Search with live internet access are not covered. Zero data retention supports HIPAA compliance without satisfying it: you still need the BAA, access controls, audit logging, and a data-flow map. None of this is provided by OpenMetal’s organization-level HIPAA posture, which covers your infrastructure, not the API call. The two are additive and both are required for regulated data. [VERIFY] current BAA scope and eligible endpoints against OpenAI’s terms.
• Air-gapped or classified workloads exclude GPT entirely. If nothing may leave the network, no GPT route qualifies. Route those workloads to an open-weight model on your own OpenMetal GPUs instead.

5. When local GPUs enter (hybrid)

The data-plane tier and the full-sovereignty tier are not mutually exclusive. A hybrid design routes the bulk of traffic, and anything too sensitive to send out, to an open-weight model running on your own OpenMetal GPU servers, behind the same egress proxy, and escalates only the hardest prompts to GPT’s API. That shrinks what leaves your boundary and trims token spend at the same time. The open-weight side of that design is our Model Deployment Fit series, where models like Llama and gpt-oss run on OpenMetal H200 and NVIDIA RTX PRO 6000 servers, each of which uses discrete GPUs rather than a pooled accelerator.

6. Running your GPT stack on OpenMetal

The reason to run this stack on OpenMetal rather than a hyperscaler is that the data-plane tier is only worth it if the boundary is real. Single-tenancy means your vector store and your logs sit on hardware no other tenant shares. Per-customer private networking means the traffic between your app, your retrieval layer, and your proxy never touches the public internet. Included egress and fixed monthly pricing mean the architecture does not penalize you for keeping data close.

There is a strategic payoff as well. Because GPT sits behind your own proxy, your data plane is provider-neutral: you can point it at GPT today, add another provider tomorrow, or fail over between them, without moving your corpus, your logs, or your guardrails. You own the substrate, and the model is a swappable backend. That is the opposite of building your whole stack inside one vendor’s managed platform.