Your AI Just Quoted One Patient's Chart to Another Patient. Nobody Was Hacked.

A twelve-physician multispecialty group I worked with last winter caught it by accident.

An OB-GYN was reviewing a discharge summary her AI scribe had drafted for a routine post-op visit. The note read cleanly until the third paragraph, which referenced an orthopedic follow-up she had never recommended. The patient had never seen orthopedics. The patient had never mentioned orthopedics.

Two appointments earlier, on the same model, in the same hour, a different patient had been referred to orthopedics.

That is AI data leakage. No hacker. No breach alert. No 500 error. No failed login. The infrastructure layer says everything is fine because by every traditional definition of fine, everything is fine. Authentication held. The API returned a 200. Latency was clean.

The AI service layer leaked a piece of one patient's record into another patient's note. The dashboard never noticed because the dashboard was never designed to.

This is the failure mode that HIPAA-covered entities, financial institutions, and law firms running multi-tenant LLM applications are quietly accumulating in production right now. With about one in eight US medical practices having now deployed an AI receptionist of some kind, per recent reporting in Healthcare IT Today citing research from The Algorithm, this exposure is no longer hypothetical for a long tail of healthcare organizations. And nobody is monitoring for it.

How Does Data Leakage Happen in LLM Applications?

The mechanism is mundane. It is not exotic. It is not an attack. It is a performance optimization gone sideways.

Most production AI services cache conversation context. Holding the recent turns of a conversation in memory means the model does not need to re-tokenize the entire history on every call. For a chat product serving one user that is a clean optimization. For a multi-tenant platform serving thousands of users on shared inference infrastructure, that same cache is a liability.

The failure starts at the seam between two systems built by different teams thinking about different problems.

The application layer handles authentication. It validates the session token, confirms the user is who they say they are, confirms they have a right to their own data. This layer is rigorous. Security teams have been hardening it for thirty years.

The AI service layer handles inference. It receives a prompt, retrieves context, generates a response. In a worrying number of production architectures, that layer inherits trust from the app layer instead of verifying it. The app layer said this request is authorized. The AI layer assumes that means the context already loaded in memory is the right context for this request.

If the context cache is keyed by anything other than a strict per-request scoping (session ID, tenant ID, patient MRN, account number), the assumption breaks the moment two requests overlap in the same inference worker. User A's last prompt is still warm in memory when User B's request lands. The model uses what is closest to hand. The response goes out.

The authentication was correct. The authorization at the inference layer was never checked.

What Is Session Bleed, Specifically?

Session bleed is when conversational context, retrieved documents, or cached intermediate state from one user's AI session leaks into another user's session. The LLM session bleed pattern is uniquely dangerous because the leaked output looks completely normal.

The response is fluent. The grammar is correct. It is responsive to the question that was asked. It contains specific, accurate-looking data. The only problem is the data belongs to someone else.

From the receiving user's perspective, they got an answer that looks helpful. They have no way to know the orthopedic referral was never theirs. AI systems present every response with the same fluent confidence whether the answer is grounded, hallucinated, or accidentally borrowed from someone else's session.

From the monitoring system's perspective, the request was handled successfully. 200 response. Latency within SLA. No error logged. No alert fired.

The failure is invisible to both ends, the user and the infrastructure. The only place it surfaces is in the response itself, and only if somebody happens to read the response carefully enough to catch it.

That is what made the OB-GYN's catch unusual. She remembered her own patients. The AI scribe was confident enough that another physician, signing fifteen of these notes in a row at the end of a long clinic day, might have signed it and moved on.

Why Your Existing Security Monitoring Cannot See This

The security stack covering most production AI deployments was assembled from tools designed for a different threat model.

• WAFs watch for injection attempts. They do not parse generated text for PHI that belongs to another patient.
• SIEMs ingest authentication logs and infrastructure events. They do not have ground truth for which patient's chart should have been used to compose which note.
• DLP scanners look for PHI leaving the perimeter. They do not flag PHI being routed between two authorized internal users when both have legitimate access to their own data.
• EHR audit logs record which records were opened by which user. They do not capture which records were loaded into an AI service's context window during inference, or whose response that context contributed to.

The leak happens in a layer the existing security stack does not instrument. Every traditional control is watching the doors. The leak is happening in the kitchen.

How Should Multi-Tenant AI Applications Handle User Isolation?

Multi-tenant LLM security is an inference-layer problem with an inference-layer fix. The four controls that matter:

Per-request context scoping. Every inference call carries an explicit user, tenant, or patient identifier that the AI service uses to load context. The service does not inherit context from an in-memory cache that happens to be warm. If the cache is keyed by anything coarser than the most specific identifier in your data model (patient MRN, matter number, account ID), it is a leakage vector waiting for production load.

Session boundary enforcement. When a session ends, every cache entry, every intermediate state object, every conversational memory blob tied to that session is invalidated immediately. No residual context survives into the next session. This is what Claire does on every voice call: the inference context is allocated fresh per call and torn down when the call ends, so the trace for the next patient on the line cannot inherit a single token from the trace for the patient who hung up thirty seconds earlier.

Data lineage tracing. Every AI response is paired with a record of which context records contributed to it and which user those records belonged to. If a response contains data the requesting user should not have seen, the lineage record makes the leak detectable forensically. Without lineage, the only way you find session bleed is the way the OB-GYN found hers, by reading carefully enough to recognize the wrong fact.

Inference-layer authorization. The app layer confirmed identity. The AI layer must independently confirm that the data about to be packed into the prompt belongs to the identified user. These are two separate checks. They are not redundant. They protect against two different failure modes.

The deeper architectural treatment of all of this lives in the multi-tenant AI architecture guide.

Is AI Data Leakage a HIPAA, GDPR, or GLBA Issue?

Yes, in all three regimes, and the absence of a hacker does not change the answer.

HIPAA. PHI from one patient disclosed to another patient, or any individual not authorized to receive it, is an unauthorized disclosure under the Privacy Rule. The Breach Notification Rule presumes a breach unless the covered entity demonstrates low probability of compromise through a four-factor risk assessment. The clock for individual, HHS, and (for breaches affecting 500 or more) media notification starts the day the disclosure is discovered, not the day it occurred. The fact that the disclosure happened via a model's cache rather than an intruder's exfiltration is irrelevant to the notification obligation. The HIPAA AI risks guide covers the operational implications. The BAA sub-processor risks guide covers how this exposure flows through your AI vendor stack.

GDPR. Personal data disclosed to an unauthorized recipient is a personal data breach under Article 4(12), regardless of the technical mechanism. Article 33 requires notification to the supervisory authority within 72 hours of becoming aware. Article 34 requires notification to data subjects when the breach is likely to result in high risk. Cached context bleed clears both thresholds for any meaningful volume of affected users. The GDPR AI compliance guide walks through the controller obligations specific to multi-tenant LLM deployments.

GLBA and state financial laws. Nonpublic personal information disclosed across customer accounts triggers state notification regimes (most aggressively, New York DFS Part 500 and California's expanded CCPA/CPRA framework). The Safeguards Rule reasonable-security obligation is not satisfied by an architecture that allows one customer's transaction history to appear in another customer's response.

In every regime the obligation attaches to the disclosure, not the cause. "It was a caching bug, not an attack" is not a defense. It is a sentence in your breach notification letter.

What to Ask Your AI Vendor Before You Go Live

If you are buying an AI scribe, an AI receptionist, an AI client-intake tool, or any multi-tenant LLM product touching regulated data, the diligence questions that matter are not the ones on the standard vendor security questionnaire.

1. How is inference context scoped per request? Specifically, what is the cache key, and is it specific enough to make cross-tenant or cross-patient context collision architecturally impossible?
2. How long does context persist after a session ends? A real answer is measured in milliseconds. An evasive answer is measured in "until the next request rotates it out."
3. Do you log every prompt with the user, tenant, or patient identifier it was scoped to? If you cannot get this log on demand, you cannot do a four-factor risk assessment when something goes wrong.
4. Have you stress-tested concurrent multi-tenant load with deliberate cross-tenant probe queries? Single-user QA will not surface this failure mode. It only appears under concurrent load with different data.
5. What is your breach-notification protocol if context bleed is discovered post-deployment? The vendors who have thought about this will have a clear answer. The vendors who have not will say "this cannot happen on our platform."

The last answer is the one to be most worried about. Anyone who tells you this cannot happen on their platform has not thought hard enough about the inference layer to know how it happens on every multi-tenant platform that has not designed against it specifically.

Preventing AI Context Leakage in Production

The fix is architectural. It cannot be patched after deployment, and it cannot be bolted on by a security team after the AI team ships.

Build user, tenant, or patient verification into the inference layer. Every inference request re-verifies which data should be in scope. No inheritance. No assumptions. No cached shortcuts.

Eliminate shared context across sessions. Within a single user's session, context caching is a legitimate optimization. Across sessions, context sharing is a leakage vector. There is no middle ground that does not eventually leak.

Implement response auditing with data lineage. Every response is logged with the specific records that fed into it. If the lineage shows User B's response was composed from data tagged to User A's MRN, the audit log surfaces the bleed before the compliance team learns about it from a patient.

Test with adversarial concurrent users. Two test accounts hitting the same inference workers at the same time with deliberately distinctive data. If the AI ever returns User B's distinctive marker in response to User A's query, you have found the bug in QA instead of in production.

If your vendor cannot tell you how their architecture prevents this in clear, technical, specific terms, the dashboard is going to be green the entire time it is happening to you.

---

Maya Chen is the voice behind Maya Builds AI, a video and podcast series on enterprise AI infrastructure for the people building and operating these systems. Three new videos a week on YouTube. The podcast lands weekly on Spotify and Apple Podcasts. For the HIPAA-specific operational implications, read the HIPAA AI risks guide.