What HIPAA Requires from the Infrastructure Running Your Healthcare AI Workloads

The regulatory frameworks governing healthcare data were built to govern human access to patient information. AI systems stepping into clinical and administrative roles don’t step out of those frameworks.

An AI model generating clinical summaries, processing diagnostic data, or handling patient communications accesses protected health information and is subject to the same HIPAA technical safeguards as any other system that touches ePHI. For most healthcare organizations, this isn’t a surprise.

What is increasingly apparent in 2026 is that the infrastructure choices underlying healthcare AI deployments have direct compliance implications, and the ones made without that in mind create audit complexity that surfaces at exactly the wrong time.

What HIPAA Requires for AI Systems Specifically

HIPAA’s Privacy and Security Rules apply to any system that accesses, processes, or stores protected health information. For AI workloads, several requirements have specific infrastructure implications.

Business Associate Agreements cover every vendor in the chain

Any AI vendor that accesses PHI on behalf of a covered entity is a Business Associate under HIPAA and must sign a BAA before processing patient data. The BAA must specifically address how the AI system interacts with PHI, what happens to data after processing, whether patient data is used for model improvement or retraining, and what security measures protect PHI within the AI pipeline. This last point matters: many AI vendors use customer data to improve their models. A healthcare organization must contractually prohibit this for any vendor handling identifiable patient data.

The minimum necessary standard applies to AI data access

HIPAA’s Privacy Rule requires that systems access only the specific PHI elements required for their intended function. An AI model that performs clinical documentation review doesn’t have a legitimate basis for accessing billing history. Healthcare organizations must evaluate what data each AI system actually requires and restrict access accordingly, which has architectural implications for how AI pipelines are designed and what infrastructure they run on.

Training AI models on PHI requires explicit patient authorization

Using identifiable PHI to train a new AI model falls outside treatment, payment, or operations, the three categories of permissible PHI use without patient consent. This means healthcare organizations building or fine-tuning AI models on their own patient populations must either obtain written authorization, use properly de-identified data, or run training in a way that satisfies a waiver of authorization under an IRB process. The infrastructure running that training needs to support the same PHI controls as any other ePHI system.

Vector embeddings of patient data are PHI

This is frequently missed in healthcare AI architecture reviews. When patient records are converted to vector embeddings for retrieval-augmented generation or semantic search, those embeddings constitute PHI under HIPAA. They carry an information relationship to the patient even without obvious identifiers. The vector store holding those embeddings must be covered by the same technical safeguards and BAA terms as the source records.

The 2026 HIPAA Security Rule Update

The 2026 HIPAA Security Rule update introduces requirements that directly affect AI infrastructure. Two changes are relevant here.

Encryption for ePHI, previously classified as an “addressable” specification that organizations could implement or document an equivalent measure for, is now required. This applies to all ePHI, including data processed by AI systems. Encryption at rest (AES-256) and in transit (TLS 1.2 or higher) for every component in the AI pipeline is now a mandatory technical safeguard, not a recommended one.

New vulnerability scanning requirements apply specifically to AI infrastructure. Organizations must implement processes to identify and remediate vulnerabilities in the systems running AI workloads that handle ePHI. For healthcare organizations running AI on shared public cloud infrastructure, this means the vulnerability management process extends to infrastructure they don’t control and can’t independently scan. For organizations running AI on dedicated infrastructure, the vulnerability scope is more clearly bounded and independently assessable.

Why Shared Cloud Multi-Tenancy Complicates HIPAA Documentation

Public cloud infrastructure can be made HIPAA-compatible. AWS, Azure, and GCP all sign BAAs and provide compliance documentation. The challenge is not that shared cloud is impermissible. The challenge is that auditing it accurately is more complex, and the documentation burden falls on the covered entity.

In a shared multi-tenant environment, physical isolation between tenants is managed by the cloud provider’s hypervisor and network controls. HIPAA compliance is a shared responsibility: the provider secures the core infrastructure, but the customer is responsible for how they configure and use it. For a HIPAA audit, the covered entity needs to document not just their own controls but the boundary between what the provider controls and what they control. That boundary is well-documented in cloud providers’ compliance documentation, but it requires ongoing attention as services and configurations change.

For AI-specific workloads, the shared responsibility model has additional dimensions. The concern about model training on customer data is addressed contractually through the BAA, but verifying it is operationally enforced is harder in a shared environment than in a dedicated one. Audit logs in shared cloud environments reflect what happened to data within the tenant’s account but not the underlying infrastructure access that the provider’s personnel could theoretically have.

For organizations under strict audit requirements or handling highly sensitive PHI, these are documentation burdens that grow with each audit cycle.

What Dedicated Private Infrastructure Changes

Running healthcare AI on dedicated private infrastructure doesn’t eliminate HIPAA obligations. It changes which parts of the compliance documentation are straightforward and which require active management.

Physical isolation is documented simply

On dedicated single-tenant hardware, there are no other tenants on the same physical servers. The PHI isolation boundary is the hardware itself. For an auditor reviewing where patient data lives and who can access the physical layer, the answer is clean.

The BAA covers infrastructure you directly control

OpenMetal signs BAAs as a HIPAA Business Associate, which means the contractual PHI protection obligations are in place at the infrastructure layer. Because OpenMetal doesn’t use customer data for any purpose beyond delivering the infrastructure service, the model-training-on-customer-data concern that affects consumer AI services doesn’t apply.

Audit logging is more granular and accessible

HIPAA requires comprehensive audit controls: logs of who accessed what, when, and what they did. On dedicated bare metal and OpenStack private cloud, audit logging is configurable at the infrastructure level with full access to the logs. The audit trail is yours, not mediated through a provider’s log export API with retention limits.

Vulnerability scanning applies to infrastructure you can independently assess

The 2026 requirement for vulnerability scanning of AI infrastructure is easier to satisfy when you have direct access to the infrastructure. On OpenMetal, vulnerability scanning of the systems running your AI workloads is operationally possible in a way that scanning infrastructure you don’t control isn’t.

Data residency is straightforward

OpenMetal operates from data centers in Ashburn VA, Los Angeles CA, Amsterdam NL, and Singapore. PHI that lives on OpenMetal infrastructure in a specific facility stays in that facility. For healthcare organizations with state-level data residency requirements or GDPR obligations on EU patient data, the data location is a known, fixed fact rather than a configuration to verify and maintain.

TDX for Maximum-Sensitivity Workloads

For healthcare AI workloads handling the most sensitive PHI, OpenMetal’s XL v4 and XL v5 servers support Intel TDX out of the box. Trust Domain Extensions provide hardware-level memory encryption that isolates running workloads from the infrastructure layer itself, meaning that even OpenMetal’s own operations personnel cannot access the plaintext contents of a running Trust Domain.

This is relevant for specific healthcare AI use cases: AI inference on identifiable diagnostic data where the organization wants hardware-level proof that the inference process is isolated, key management infrastructure for healthcare AI systems where the key material must be protected at the hardware layer, and regulated research environments where the data governance requirements demand isolation proof that software controls alone can’t provide.

TDX is not a requirement for HIPAA compliance. It is an additional layer for workloads where the threat model includes the infrastructure operator and where the compliance or contractual requirements need hardware-level isolation attestation. For healthcare AI teams whose compliance posture requires that level of assurance, OpenMetal is one of the few infrastructure providers where it is available on production-grade dedicated hardware.

What Private Infrastructure Doesn’t Solve

The same caveat that applies to every compliance-adjacent infrastructure conversation applies here: infrastructure is one component of a HIPAA compliance program, not the program itself.

A covered entity running healthcare AI on dedicated private infrastructure still needs to conduct and document risk analyses, implement workforce training, maintain policies and procedures, execute BAAs with all vendors in the chain, and address the administrative and physical safeguard requirements that no infrastructure provider can satisfy on their behalf. HHS OCR resolved 21 HIPAA enforcement cases in 2025, with 76% including penalties for risk analysis failures. The most common deficiency is documentation of risk analysis, not inadequate infrastructure controls.

OpenMetal’s role is to provide infrastructure with a clean compliance posture: HIPAA Business Associate agreement available, ISO 27001:2022 framework, SOC 2 in pursuit, single-tenant architecture, documented data locations, and hardware-level confidential computing available for the workloads that require it. Building a healthcare AI program on that infrastructure is meaningfully simpler to document than building it on shared cloud with a complex shared-responsibility boundary. But the compliance program that makes use of that infrastructure still requires dedicated organizational investment.

Healthcare AI teams building out their infrastructure approach can start with a PoC evaluation to validate that the infrastructure meets their specific workload and documentation requirements before committing.

Frequently Asked Questions

Does HIPAA prohibit running healthcare AI on public cloud infrastructure?

No. Public cloud infrastructure can be made HIPAA-compatible when a BAA is in place and the appropriate technical safeguards are implemented. The challenge is not permissibility but audit complexity: the shared-responsibility model requires ongoing documentation of the boundary between provider controls and customer controls, which adds compliance overhead that grows with each audit cycle.

Are vector embeddings of patient data considered PHI under HIPAA?

Yes. Vector embeddings that carry an information relationship to an identifiable patient are protected health information under HIPAA, even if they don’t contain obvious identifiers. Healthcare organizations building RAG systems or semantic search on patient data must apply the same PHI safeguards to their vector stores as to the source records.

Can a healthcare organization train an AI model on patient data?

Training an AI model on identifiable PHI generally requires explicit written patient authorization because it falls outside treatment, payment, and operations. Alternatives include properly de-identified data (which is no longer PHI under HIPAA and can be used freely), synthetic data generation, or authorization through a formal IRB waiver process. The infrastructure running that training must apply PHI-equivalent controls until de-identification is complete.

What changed in the 2026 HIPAA Security Rule update for AI infrastructure?

The 2026 update made encryption of ePHI mandatory (previously “addressable”) and introduced specific vulnerability scanning requirements for AI infrastructure. Both changes apply to the technical environment where AI systems process patient data, not just the AI application layer.

What is Intel TDX and why does it matter for healthcare AI?

Intel Trust Domain Extensions (TDX) is a hardware-based confidential computing feature that isolates running workloads in encrypted Trust Domains. Even the infrastructure operator cannot access the plaintext memory of a running Trust Domain. For healthcare AI workloads where the threat model includes the infrastructure layer, TDX provides hardware-level isolation attestation that software controls cannot replicate. OpenMetal’s XL v4 and XL v5 servers support TDX out of the box.

Does OpenMetal sign BAAs for healthcare customers?

Yes. OpenMetal operates as a HIPAA Business Associate and signs BAAs for customers running workloads that process protected health information. OpenMetal’s ISO 27001:2022 framework and single-tenant architecture support the documentation requirements that healthcare organizations need for HIPAA audits.