Healthcare Industry

Shadow AI in Healthcare: PHI Exposure You Cannot See

Clinicians paste patient notes into ChatGPT. Coders use AI for ICD lookups. Researchers upload datasets to model-training tools. Your EHR audit trail does not catch any of it. A DNS log audit does.

Why Healthcare Shadow AI Is a HIPAA Emergency

Every unsanctioned AI tool that touches patient data is a potential breach notification event. Unlike other industries, healthcare has strict federal penalties and mandatory 60-day disclosure timelines.

Clinical Staff Copy-Paste

Physicians and nurses paste discharge summaries, lab results and medication lists into AI chat tools to draft notes faster. Each paste sends PHI to a third-party server with no BAA in place.

Medical Coding AI

Coders use AI-powered ICD-10 and CPT lookup tools to speed up claims processing. These tools often require pasting procedure narratives that contain patient identifiers.

Research Data Uploads

Researchers upload de-identified datasets to AI analysis platforms. But de-identification is rarely perfect, and the tool's terms may permit training on uploaded data.

Administrative Summarizers

Front-desk and billing staff use AI summarizers for prior authorization letters, appeal drafts and patient communications. Each contains names, DOBs and diagnosis codes.

AI Diagnostic Assistants

Clinicians query AI tools for differential diagnosis suggestions, uploading symptoms, vitals and imaging descriptions. No BAA, no audit trail, no HIPAA compliance.

Training and Education

Medical residents and students use AI to study case files, sometimes uploading real patient scenarios from teaching hospitals to get AI-generated feedback.

HIPAA Control Mapping for Shadow AI

A shadow AI audit maps directly to HIPAA Security Rule requirements. Every finding in your report addresses a specific safeguard obligation.

HIPAA RequirementSectionShadow AI RiskWhat the Audit Produces
Access Controls164.312(a)AI tools accessed without authentication controlsList of AI tools per user group with access frequency
Audit Controls164.312(b)No logging of PHI sent to AI servicesDNS-level activity log for every AI domain contacted
Transmission Security164.312(e)PHI transmitted to AI servers without encryption verificationProtocol analysis and encryption status per AI tool
Business Associate Agreements164.314(a)AI vendors used without BAA in placeVendor list with BAA availability flag per tool
Risk Analysis164.308(a)(1)AI tool risks not included in risk assessmentRisk-scored AI tool inventory with remediation priority
Workforce Training164.308(a)(5)Staff unaware of AI tool PHI risksUsage patterns by department for targeted training
Device and Media Controls164.310(d)AI browser extensions on workstationsExtension and plugin detection from network signatures

What Your Healthcare Audit Report Shows

A real excerpt from a shadow AI audit of a 400-bed hospital system. Names and specifics redacted.

SHADOW AI AUDIT REPORT - HEALTHCARE FACILITY
Scan Period14 days (DNS + proxy)
Total AI Tools Found34 unique AI services
Tools with BAA6 of 34
Tools Training on Input11 of 34 (confirmed)
PHI Exposure RiskHIGH - 28 tools lack BAA
TOP FINDINGS
ChatGPT (Free Tier)1,247 queries - trains on input, no BAA
Claude.ai89 queries - does not train, BAA available
Scribe AI412 queries - clinical note generation, no BAA
Perplexity203 queries - trains on input, no BAA
Otter.ai67 sessions - records meetings, no BAA
RISK DISTRIBUTION
Critical (no BAA + trains on input)
High (no BAA, opt-out available)
Managed (BAA in place)

Real-World Healthcare Shadow AI Scenarios

Each scenario is drawn from patterns we observe across healthcare audit engagements.

Scenario 1: The Discharge Summary

An attending physician pastes a full discharge summary into ChatGPT to reformat it for a patient portal. The summary contains the patient's name, DOB, diagnosis codes, medications and attending notes. OpenAI's free tier trains on this input. This is a HIPAA breach exposing PHI to a vendor without a BAA.

Scenario 2: The Billing Appeal

A billing specialist uses an AI writing tool to draft a payer appeal letter. The letter contains the patient's name, insurance ID, procedure codes and clinical justification. The AI tool stores the input in its training dataset. The health system has no record this happened.

Scenario 3: The Research Upload

A clinical researcher uploads a CSV of "de-identified" patient data to an AI analysis tool. The dataset contains rare disease combinations and zip codes that make re-identification possible. The AI vendor's terms allow training on uploaded data and sharing aggregate outputs.

Scenario 4: The Meeting Transcription

A department head uses an AI meeting recorder during a case review. The recording captures patient names, conditions and treatment decisions. The transcription tool sends audio to cloud servers for processing. No BAA exists. The transcript is stored indefinitely.

Healthcare Shadow AI Remediation Framework

A four-phase approach to moving from invisible risk to managed AI governance.

Phase 1

Discover

Run the shadow AI audit. Upload 14 days of DNS and proxy logs. Get a complete inventory of every AI tool accessed from your network.

Phase 2

Classify

Score each tool for PHI exposure risk. Flag tools that train on input, lack BAAs, or store data outside the US. Map findings to HIPAA requirements.

Phase 3

Remediate

Block critical-risk tools. Negotiate BAAs for tools worth keeping. Update your acceptable use policy. Deploy targeted training for high-usage departments.

Phase 4

Monitor

Run quarterly audits to detect new AI tool adoption. Track policy compliance by department. Feed results into your HIPAA risk assessment cycle.

Shadow AI Usage by Department

Which departments use the most unsanctioned AI tools, and what types they reach for.

DepartmentCommon AI ToolsTypical Use CasePHI Risk Level
Emergency MedicineChatGPT, differential diagnosis toolsSymptom lookup, note draftingCritical
Medical Records / HIMAI coding assistants, summarizersICD-10 coding, chart abstractionCritical
Billing and Revenue CycleAI writing tools, appeal generatorsPayer appeals, denial managementHigh
RadiologyAI image analysis, report toolsImpression drafting, comparisonCritical
NursingChatGPT, clinical reference toolsCare plan drafting, patient educationHigh
ResearchAI analysis platforms, data toolsDataset analysis, literature reviewHigh
AdministrationAI transcription, meeting toolsMeeting notes, presentationsMedium
IT / InformaticsAI code assistants, chatbotsEHR customization, scriptingLow

The Cost of a Healthcare AI Data Breach

Healthcare has the highest average breach cost of any industry. Shadow AI creates breach vectors that traditional security tools miss entirely.

$10.93 Million

Average cost of a healthcare data breach in 2023, the highest of any industry for the 13th consecutive year. Shadow AI creates new breach vectors outside your security perimeter.

60-Day Window

HIPAA requires breach notification within 60 days of discovery. If shadow AI tools expose PHI, the clock starts when you find out. A proactive audit finds the exposure before it becomes a reportable incident.

OCR Enforcement

The HHS Office for Civil Rights has increased enforcement actions. Penalties range from $100 to $50,000 per violation, with annual maximums of $1.5 million per violation category.

Healthcare Shadow AI Audit Pricing

One log upload. One report. Complete visibility into every AI tool touching your network.

View All Plans Start Free Audit
Operational map

Where shadow AI appears in healthcare work

AI adoption rarely begins as a formal platform project. It begins when a person finds a faster way to finish a familiar task. For clinicians, researchers, revenue-cycle staff, administrators and technology teams, that convenience can put protected health information, clinical notes, images, claims, research data and patient identifiers into services the organization has not assessed.

Clinical documentation

Staff draft notes, discharge instructions or referral letters. A useful prompt can still disclose diagnoses, medications and identifiers.

Medical imaging

Users test image enhancers or diagnostic assistants outside the approved imaging environment, creating an uncontrolled copy of patient data.

Revenue cycle

Claims teams use AI to explain denials or rewrite appeals. Claim numbers and clinical context can make supposedly de-identified text identifiable.

Research

Investigators summarize papers, analyze datasets or generate code. Unpublished results and participant information may cross contractual boundaries.

Patient communications

Contact centers and care coordinators draft replies in consumer tools, where pasted conversation history may contain extensive health information.

Technology support

IT teams paste logs and screenshots into assistants. Those artifacts can contain patient names, tokens and internal system details.

Read the signal carefully. A domain match proves that an identity or network resolved an AI service during the audit window. It does not reveal a prompt or prove that regulated data was uploaded. Use the result to focus interviews, vendor review and technical investigation where they are most likely to matter.
Decision framework

Turn an inventory into defensible decisions

A useful program does more than count domains. It connects observed use to data, purpose, identity and vendor terms. That context lets privacy, security, clinical governance, compliance and departmental leadership choose a proportionate action and explain it later.

QuestionWhy it mattersPractical outcome
Is the tool approved?Approval confirms that ownership, terms and intended use were reviewed. A familiar brand is not evidence that the account or feature in use is sanctioned.Label the service approved, restricted, under review or blocked. Record the account tier and approved purpose, not only the vendor name.
What data could reach it?The same assistant can be low risk for public research and high risk for protected health information, clinical notes, images, claims, research data and patient identifiers.Define prompt examples by data class. Give users concrete red lines and a safe route for the useful workflow.
What do the terms say?Training, retention, subprocessors, location, deletion and administrative controls vary by plan and can change over time.Capture a dated vendor decision. Recheck material services on renewal and whenever the report shows a new domain or product.
Who is using it?Ten hits from one lab, practice or branch call for a different response than ten hits spread across public-research roles.Start with the highest-risk identity cluster. Ask about the task before deciding whether to coach, migrate, investigate or block.
Does an existing tool cover the need?Shadow adoption often signals that an approved platform is unavailable, difficult to use or poorly communicated.Remove friction, publish the approved route and measure whether users migrate during the next comparable audit window.
Six-step playbook

Run the first review without creating panic

The strongest first audit is narrow enough to explain and repeat. Use a normal operating period, preserve the source details and state the limits of DNS or proxy evidence in the report.

Choose a representative window

Start with 14 to 30 days that avoid a shutdown, migration or major outage. Record the dates, networks, identity coverage and exclusions. That scope statement makes comparisons honest and prevents a partial export from being mistaken for an enterprise census.

Export the minimum useful fields

From hospital DNS, secure web gateways, remote-access infrastructure and managed endpoint telemetry, retain timestamp, hostname, action and the most useful identity field available. Do not add URL paths, request bodies or prompt content. Inventory work needs enough attribution to act, not the maximum amount of personal data.

Validate the largest clusters

Remove known test traffic, shared resolvers and infrastructure identities before interpreting user counts. Look for repeated use over several days and clusters within a team. A single lookup may be an email link or background process; a stable pattern deserves a conversation.

Add business context

Meet the relevant team with neutral questions: what task is being completed, what information is entered, which account tier is used and why the approved route does not work. Document the answer beside the technical observation so reviewers can distinguish evidence from inference.

Assign a proportionate action

Block services built for abuse or those with unacceptable terms. Migrate useful work to an enterprise product when controls can make it safe. Coach occasional low-risk users, and escalate suspected disclosure through the existing incident process rather than inventing an AI-only procedure.

Re-run and close the loop

Use the same source and similar window after remediation. Compare tool prevalence, active identities and sanctioned share. A decline supports closure; persistent traffic can reveal browser extensions, mobile devices, alternative domains or a workflow the replacement failed to solve.

Shared ownership

Give every function a clear part in the control

Shadow AI crosses technical, contractual and operational boundaries. A central policy works only when each function knows which decision it owns and where to send an exception.

Security and IT

Maintain discovery coverage, investigate technical anomalies and operate access controls. Security should explain what the logs prove and where visibility ends. It also checks whether sanctioned services support identity, logging, retention and configuration standards.

Privacy and legal

Translate observed workflows into data, confidentiality and notification questions. Review terms for training, retention, location and subprocessors. When facts suggest a disclosure, use the established legal and privacy assessment process rather than deciding from a domain count.

Procurement and vendor risk

Prioritize services that employees actually use. Confirm the exact plan, contracting entity and product features because one vendor can offer several materially different data practices. Preserve the assessment date and schedule a review for important services.

Business owners

Explain the task, data and reason for adoption. Decide whether the workflow is valuable enough to sanction, redesign or replace. Owners also nominate local guidance, test approved alternatives and confirm that remediation works in daily practice.

People and communications

Give employees examples they recognize, a simple approved-tool directory and a route to request new uses. Communications should avoid implying that DNS reveals prompts. Training is more credible when it solves the workflow that led users to the tool.

Internal audit and assurance

Test whether scope is documented, findings have owners, exceptions expire and follow-up runs verify change. Assurance can sample the chain from source export to final disposition without needing access to every identity in the operational report.

Evidence pack

What to preserve for audit, legal and leadership review

In healthcare, the relevant obligations can include HIPAA Privacy and Security Rules, HITECH breach duties, GDPR health-data rules, medical-device quality controls, records-retention requirements. A domain inventory does not satisfy those frameworks by itself. It supplies a dated factual layer that existing governance processes can use.

Scope statement

Identify the log source, dates, included networks and available identity types. List known blind spots such as unmanaged mobile data, direct IP connections or sites that use a different resolver.

This keeps later readers from giving the numbers more certainty than the collection method supports.

Observed-tool register

Keep the service name, matched domains, category, first and last observation, identities and sanctioned status. Store dated vendor-term research beside the observation rather than silently overwriting old decisions.

The result becomes a practical subset of the wider AI system inventory.

Decision record

For each material tool, name an owner, intended use, permitted data, account requirement and disposition. Include the reason for approval, restriction or blocking in language a business user can follow.

A short rationale is more useful than a color with no explanation.

Remediation evidence

Attach tickets, procurement decisions, configuration changes, user guidance and incident references where applicable. Link each artifact back to the finding it addresses.

This traceability shows that discovery led to controlled action rather than a report that was filed and forgotten.

Exception register

Some teams need a restricted tool temporarily. Record the business reason, data limits, accountable owner, compensating controls and expiry date.

Time-bound exceptions prevent urgent workarounds from becoming permanent, invisible policy.

Follow-up comparison

Preserve a comparable later report and explain major changes in scope. Track whether targeted identities moved, whether alternative domains appeared and whether approved services gained adoption.

Trend evidence makes the control testable instead of merely aspirational.

Worked example. A hospital audit finds a transcription domain used by a small cluster of outpatient identities. Privacy and clinical operations review the pattern together and learn that staff were recording virtual follow-ups to speed documentation. The organization pauses that workflow, validates whether protected information was processed, selects an approved service with suitable terms and uses the next audit to confirm migration.
Management view

Metrics that show control is improving

A large raw hit count is rarely a useful board metric because DNS caching and automated traffic distort volume. Favor stable measures tied to coverage, decisions and follow-through.

Inventory coveragePercentage of in-scope networks and identities represented in the export, with missing locations named.
Sanctioned adoptionShare of active AI identities using approved services during the period, tracked against the same population.
Unreviewed servicesNumber of newly observed tools with no owner or documented decision, plus their age since discovery.
High-risk identitiesDistinct identities reaching services marked for restriction or blocking, counted without implying confirmed disclosure.
Time to dispositionMedian days from first observation to approve, restrict, migrate, block or close as benign technical traffic.
Remediation persistencePercentage of targeted findings absent in the next comparable run, with bypasses investigated separately.
Program questions

Questions Healthcare teams ask

Does a DNS match prove that sensitive information was uploaded?

No. It proves that a covered identity or network resolved a domain during the selected period. Confirm the workflow through interview, endpoint evidence or the vendor’s administrative logs before making a disclosure finding.

Why not block every public AI service immediately?

Immediate blocking can interrupt legitimate work and push users to personal devices or less visible services. Discovery first lets the organization protect urgent risks, preserve useful workflows and provide an approved alternative users will actually adopt.

Can this replace vendor risk assessment?

No. The inventory tells vendor risk which services are actually present and which deserve priority. Procurement, privacy, legal and security still need to assess terms, controls, subprocessors, location, retention and the intended data.

How often should the organization repeat the audit?

Quarterly is a practical baseline for a stable environment. Monthly review suits fast adoption, active remediation or higher-risk groups. Repeat after major policy, network or approved-tool changes, using comparable scope whenever possible.

What if the logs identify a device rather than a person?

Keep the device identity and map it through the asset system only when investigation requires it. Network-level attribution is still useful for locating a site or function, and the report should state that it does not represent a unique-user count.

How should employee communications describe the review?

Explain the business purpose, data sources, fields used, retention, who can see results and how findings will be validated. Clear notice supports trust and prevents a domain observation from being presented as surveillance of prompt content.

What belongs on the approved AI list?

Name the product and account tier, approved purposes, permitted data classes, required settings, owner and review date. A vendor-level approval is too broad when consumer and enterprise plans have different training and administrative controls.

When does a finding become an incident?

Use the organization’s existing incident criteria. Escalate when validation indicates that protected information entered an unauthorized service, credentials were exposed or contractual and regulatory duties may apply. The domain match alone is a lead, not the final conclusion.

Related Resources

Healthcare Shadow AI FAQ

Does a shadow AI audit access patient data?
No. The audit analyses DNS query logs and proxy access logs only. These logs contain domain names and timestamps. No patient data, no PHI, no clinical content is accessed or transmitted. The audit identifies which AI tool domains were contacted, not what was sent to them.
How does this differ from our EHR audit trail?
Your EHR audit trail tracks who accessed which patient record inside the EHR system. It does not track what happens after a clinician copies text from the EHR and pastes it into a browser-based AI tool. A DNS log audit catches the network-level activity that EHR audit trails miss entirely.
Can we run the audit without involving IT?
You need a DNS or proxy log export, which typically requires IT or network team involvement. However, the audit itself runs outside your network. You upload the log file, we process it, and you get the report. No agents are installed on endpoints or servers.
What if we find PHI exposure through shadow AI?
The audit identifies which AI tools were accessed and their risk profiles (training policy, BAA availability, data storage location). If you find high-risk tools were used by clinical departments, you can assess PHI exposure risk and determine whether a breach investigation is warranted under HIPAA 164.402.
How often should a healthcare organisation run a shadow AI audit?
We recommend quarterly audits aligned with your HIPAA risk assessment cycle. AI tool adoption changes rapidly. New tools appear weekly, and staff adoption patterns shift after policy changes or training events. Quarterly audits catch drift before it becomes exposure.
Does the audit cover telehealth platforms?
Yes. If a telehealth platform uses AI features (transcription, summarization, chatbot triage), those AI service domains appear in your DNS logs. The audit identifies them alongside standalone AI tools, so you get a complete picture of AI usage across in-person and virtual care.
Sector list

Want the list without running an audit?

Every AI tool with a verdict for healthcare, as a CSV and ready block lists, from our sister site AI Tools Blocklist. One-time $790.

Find Every AI Tool Touching Your Healthcare Network

Upload your DNS or proxy logs and get a HIPAA-mapped shadow AI inventory with PHI risk ratings, BAA status and department-level usage patterns.

Start Your Free Audit
View pricing plans →