A HIPAA-compliant AI application uses protected health information (PHI) only for permitted purposes and only to the minimum necessary extent, runs on services covered by a business associate agreement (BAA), applies the HIPAA Security Rule’s administrative, physical and technical safeguards across every component that touches PHI, including prompts, retrieval indexes, logs and model outputs, and keeps clinicians or staff accountable for decisions. No AI model is “HIPAA certified” on its own; compliance comes from how the whole system is designed, contracted and operated. This guide sets out a reference architecture, the safeguards that matter most for AI and a BAA checklist. It is technical guidance, not legal advice.
Key takeaways
- Every vendor that creates, receives, stores or transmits PHI for you, including cloud and AI model providers, needs a BAA.
- Treat prompts, embeddings, vector indexes, caches and logs as PHI stores, with the same safeguards as the database.
- Apply the minimum necessary standard: send the model only the PHI the task needs, and de-identify where you can.
- Confirm in writing that PHI is not used to train the provider’s models and how long it is retained.
- Keep a clinician or authorised staff member responsible for clinical and coverage decisions.
What HIPAA requires of AI applications
HIPAA applies to covered entities, such as healthcare providers, health plans and clearinghouses, and to their business associates. Three rules shape an AI application:
- The Privacy Rule limits how PHI may be used and disclosed, including the minimum necessary standard.
- The Security Rule requires administrative, physical and technical safeguards for electronic PHI, starting with a documented risk analysis.
- The Breach Notification Rule requires notification when unsecured PHI is compromised.
An AI feature does not change these obligations; it adds new places where PHI flows. Each must be identified in the risk analysis.
A HIPAA-compliant AI reference architecture
Clinical data typically arrives through standard interfaces such as HL7 FHIR. See our guide to healthcare FHIR API integration. An AI gateway sits between the application and the model, enforcing the minimum necessary rule, filtering outputs and recording an audit trail. Retrieval indexes and caches live inside the same protected environment as the source data.
| Safeguard | Security Rule area | How it applies to the AI layer |
|---|---|---|
| Risk analysis | Administrative | Includes prompts, embeddings, indexes, logs and model providers as PHI flows |
| Access control | Technical | Unique user IDs; retrieval returns only records the user may access |
| Audit controls | Technical | Logs of who asked what, which records were retrieved and what was returned |
| Integrity | Technical | AI outputs are drafts until reviewed; source records are not altered silently |
| Transmission security | Technical | Encryption in transit to every service, private network paths where possible |
| Workforce training | Administrative | Staff know what may be entered into AI tools and how to verify outputs |
| Contingency planning | Administrative | Clinical workflows continue if the AI service is unavailable |
Safeguards specific to AI applications
- Prompt and output logging. Logs often contain PHI. Encrypt them, restrict access and set retention periods.
- Vector indexes and caches. Embeddings derived from PHI should be treated as PHI. Keep them in the protected environment with access filtering.
- Model provider terms. Use services offered under a BAA, such as those covered by Microsoft’s HIPAA/HITECH offering, and confirm data use, retention and abuse-monitoring settings.
- Guardrails. Filter outputs for PHI that should not be shown to the requesting user, and block unsafe requests. See LLM guardrails explained.
- Prompt injection. Clinical documents can contain text that tries to manipulate the model. Test against the OWASP Top 10 for LLM Applications.
- Human review. Summaries, codes and triage suggestions are reviewed by qualified staff before they affect care or claims.
De-identification and the minimum necessary standard
Where a task does not need identities, remove them. HIPAA recognises two de-identification methods: Safe Harbor, which removes 18 specified identifiers, and Expert Determination, where a qualified expert confirms the re-identification risk is very small. HHS explains both in its de-identification guidance. De-identified data is not PHI, which simplifies analytics, testing and model evaluation.
When identities are needed, send only the fields the task requires. A discharge-summary assistant may need the clinical notes for one encounter, not the patient’s full record.
BAA checklist for AI vendors
| Question | What to confirm |
|---|---|
| Is the specific service covered? | The AI service, region and features you use are listed in the BAA scope |
| Is PHI used for training? | PHI is not used to train or improve the vendor’s models |
| How long is data retained? | Retention for prompts, outputs and logs, and how deletion works |
| Who can access the data? | Conditions for vendor personnel access, including abuse monitoring |
| Where is it processed? | Regions and sub-processors |
| How are breaches reported? | Notification obligations and timelines to you |
| What happens at termination? | Return or destruction of PHI |
HHS publishes sample business associate agreement provisions as a reference.
Delivering a HIPAA-compliant AI application
- Define the use case and the PHI it truly needs.
- Update the risk analysis to include every AI component and data flow.
- Select BAA-covered services and configure data use and retention.
- Build inside the protected environment, with the AI gateway, permission-aware retrieval and audit logging.
- Test with de-identified data, then validate with authorised users on real data in a pilot.
- Train staff and keep human review in the workflow.
- Monitor and review logs, access and output quality continuously.
For broader healthcare AI patterns, see AI in healthcare use cases and architecture. For US teams, our USA engagement model plans HIPAA requirements with you per project.
Common questions
Can we use generative AI with patient data under HIPAA?
Yes, when the AI service is covered by a BAA, PHI use is limited to permitted purposes and the minimum necessary, and the Security Rule safeguards apply to every component that handles PHI.
Is any AI model HIPAA certified?
There is no official HIPAA certification for models or products. Compliance depends on contracts, configuration, safeguards and how the system is operated.
Are embeddings of medical records PHI?
Treat them as PHI. Embeddings derived from records can relate to identifiable individuals, so they need the same safeguards as the source data.
Do we need a BAA with our AI model provider?
Yes, if the provider will create, receive, maintain or transmit PHI on your behalf. Without one, do not send PHI to the service.
Can AI make clinical decisions?
AI can support decisions, but qualified clinicians should review and remain responsible for clinical judgments, consistent with your organisation’s policies and applicable regulations.
Build healthcare AI with privacy safeguards built in
Share your use case and the clinical systems involved. Our healthcare and AI/ML development teams will plan the safeguards with your compliance team and build them into the architecture from day one.