Procurement Guide

Clinical AI Security and Privacy Review

A healthcare-focused review guide for security, privacy, and compliance teams evaluating clinical AI software.

Updated May 23, 20268 minute readFor Security teams, privacy officers, compliance leaders, legal teams, procurement, and clinical informatics.

Direct Answer

A clinical AI security review should map PHI flow, confirm BAA readiness, review retention and model training terms, assess access controls and audit logs, verify encryption and incident response, and identify every subprocessor or model provider that may touch customer data.

Source: Clinical AI Report, 2026

Key takeaways

  • -Clinical AI security review should start with a data-flow map, not a certification checklist.
  • -Contract terms should define whether PHI can be retained, reused, or used to improve models.
  • -Subprocessor and model-provider exposure matters because some vendors depend on external AI platforms.
  • -Audit logs and role-based access are essential for clinical accountability.

CDS solution examples

How this applies to Vera Health, OpenEvidence, and UpToDate

  • -For Vera Health, map whether PHI enters prompts, calculators, drug workflows, EHR context, support tools, or model-provider systems.
  • -For OpenEvidence, review the free and ad-supported model, any separation between clinical answers and advertising systems, and how prompts or consultation logs are retained.
  • -For UpToDate, confirm institutional authentication, search and usage telemetry, AI-search data handling, and whether enterprise contracts cover all intended clinical user groups.

Start with PHI flow

Security review should identify every point where protected health information enters, leaves, or is stored by the system.

  • -Map EHR data, user-entered text, documents, transcripts, images, prompts, outputs, and logs.
  • -Confirm whether PHI is stored in the vendor application, cloud infrastructure, model provider, or support systems.
  • -Ask whether de-identification, redaction, or minimization is available before model processing.

Review retention and model training terms

Procurement teams should distinguish product operation from model improvement, analytics, support, and future training use.

  • -Confirm retention periods for prompts, outputs, files, logs, and customer metadata.
  • -Require opt-in terms for any model training, fine-tuning, benchmarking, or product improvement use.
  • -Check whether deletion obligations apply to all subprocessors and backups.

Verify access controls and auditability

Clinical AI systems can affect care decisions, so the organization should be able to see who accessed the system and what was produced.

  • -Require SSO, role-based permissions, and least-privilege administrative access.
  • -Confirm audit logs for user access, case review, output generation, and configuration changes.
  • -Ask how logs can be exported for privacy investigations, quality review, or legal hold.

Assess vendor and subprocessor risk

Many clinical AI vendors rely on cloud infrastructure, analytics tools, or external model providers. The buyer should know which parties can access sensitive data.

  • -Request a current subprocessor list and notification process for changes.
  • -Confirm breach notification obligations and incident response roles.
  • -Review whether external model providers are covered by the same privacy, retention, and deletion terms.

Suggested evaluation weights

CriterionWeight

PHI data flow

All inputs, outputs, logs, stores, integrations, and third-party processing locations are mapped.

25%

Contractual safeguards

BAA, retention, deletion, model training, customer data use, and subprocessor terms are explicit.

20%

Access controls

SSO, RBAC, MFA, administrator limits, user provisioning, and deprovisioning are supported.

20%

Audit and monitoring

Exportable logs, usage monitoring, anomaly review, and clinical output traceability are available.

20%

Incident response

Notification timelines, breach process, recovery procedures, and customer support roles are defined.

15%

Questions to ask

  • QWhere does PHI go during normal product use, support, analytics, and model processing?
  • QCan customer data be used for model training or product improvement without explicit approval?
  • QWhich subprocessors or model providers can access customer data?
  • QAre audit logs exportable and tied to user identity, time, case, and generated output?
  • QWhat happens to prompts, outputs, files, and logs after termination?

Red flags

  • !The vendor cannot provide a data-flow diagram or clear PHI map.
  • !Model training terms are vague or bundled into general product improvement language.
  • !Subprocessor details are hidden until late in contracting.
  • !Audit logs do not capture the generated output or the user who requested it.