Vendors now attach an AI label to note drafts, document extraction, scheduling, and policy search. The useful question is narrower: what task does this feature perform, what evidence exists for that task, who reviews the output, and where does the data go. The privacy, Part 2, device, and certification rules those answers have to satisfy are in AI in Behavioral Health: The Rules That Apply.
The task
Ask for one sentence that names the input and the output. “It drafts a progress note from the encounter the clinician selects, and the clinician edits it before signing” is a task. “It makes documentation smarter” is not. If the feature suggests a diagnosis, a medication, or a level of care, ask whether FDA treats that function as a device.
FDA’s January 2026 guidance explains the Non-Device clinical decision support criteria in section 520(o)(1)(E) of the Federal Food, Drug, and Cosmetic Act, and it states that FDA’s existing digital health policies continue to apply to software functions that meet the definition of a device, including those intended for use by patients or caregivers [1]. State licensing boards set their own practice rules. This checklist does not treat “FDA and licensing boards forbid autonomous decisions” as a single rule.
The evidence
Ask what was measured, on which records, and against what comparison. A vendor that cannot describe the measurement has not given you a result you can use. Do not accept an accuracy range that is not tied to a published study of the task you are buying.
A Health IT Module certified to 45 CFR 170.315(b)(11) must support source attributes for evidence-based decision support interventions and for predictive ones [2]. When the developer supplies either kind as part of the module, the module must enable a limited set of identified users to access complete and up-to-date plain-language descriptions of those source attributes [2]. Predictive interventions supplied by the developer must be subject to analysis of potential risks and adverse impacts associated with validity, reliability, robustness, fairness, intelligibility, safety, security, and privacy; to practices that mitigate those risks; and to policies and implemented controls for governance, including how data are acquired, managed, and used [2]. Those certification duties are about transparency of the intervention. They are not a performance score for your patient population.
The human review
For any output that will enter the legal medical record or influence a clinical decision, name the person who must review it and what happens if they reject it. A module certified to § 170.315(b)(11) provides an intervention while a user is interacting with the technology. A limited set of identified users can select, meaning activate, electronic decision support interventions, in addition to drug-drug and drug-allergy contraindication checking, including evidence-based interventions that use data expressed in the standards in § 170.213 and predictive interventions that use any data expressed in those standards [2]. That selection is not the same thing as a clinician’s signature.
The data flow
Map each hop: the program’s system, the vendor, and any subcontractor that stores or processes the record. A covered entity may disclose protected health information to a business associate, and may allow that associate to create, receive, maintain, or transmit the information on its behalf, if it obtains satisfactory assurance that the associate will appropriately safeguard the information [4]. That assurance must be documented through a written contract or other written agreement or arrangement with the business associate, as 45 CFR 164.504(e) describes [3, 4]. A covered entity is not required to obtain that assurance from a business associate that is a subcontractor [4]. A business associate may disclose protected health information to a business associate that is a subcontractor, and may allow that subcontractor to create, receive, maintain, or transmit the information on its behalf, if the business associate obtains satisfactory assurances, as 45 CFR 164.504(e)(1)(i) describes, that the subcontractor will appropriately safeguard the information [4]. The business associate contract must provide that the business associate will ensure any subcontractor that creates, receives, maintains, or transmits protected health information on its behalf agrees to the same restrictions and conditions that apply to the business associate with respect to that information [4]. The requirements in 45 CFR 164.504(e)(2) through (e)(4) also apply to the contract or other arrangement required by 45 CFR 164.502(e)(1)(ii) between a business associate and a business associate that is a subcontractor [3, 4]. A business associate may use or disclose the information only as permitted or required by its business associate contract or other arrangement under 45 CFR 164.504(e), or as required by law [4]. The contract may not authorize the business associate to use or further disclose the information in a manner that would violate 45 CFR part 164, subpart E, if a covered entity did it, except that the contract may permit the business associate to use and disclose the information for its proper management and administration, as 45 CFR 164.504(e)(4) describes, and to provide data aggregation services relating to the covered entity’s health care operations [4]. Sending a record to a vendor outside the program’s own network can be lawful. When that vendor is a business associate creating, receiving, maintaining, or transmitting the information on the covered entity’s behalf, it is not lawful without the agreement and the limits the agreement sets. A disclosure the Privacy Rule permits without a business associate, including a disclosure to another provider for treatment, does not use that agreement [4]. Part 2 adds a consent analysis for substance use disorder records; that analysis is in the companion post.
NIST’s AI Risk Management Framework 1.0, released January 26, 2023, is intended for voluntary use to better manage risks to individuals, organizations, and society associated with artificial intelligence, and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems [5]. As of September 2026, NIST states that AI RMF 1.0 is being revised, and it has published a separate generative AI profile (NIST AI 600-1, July 26, 2024) [5]. The framework is a way to organize questions about a feature. It is not a certification, and it does not rank which feature to buy first.
What not to take from a demo
A demo can show that a feature extracts fields from a document or sorts transactions. It cannot show that the extraction removes the need for staff review, that a model trained on one organization’s charts will outperform another, or that a feature will pay for itself. Ask for the task, the evidence, the reviewer, and the data flow, and keep the feature off until those four answers are written down.
Sources
- Clinical Decision Support Software (guidance, January 2026), FDA. Accessed September 28, 2026.
- 45 CFR 170.315, Certification criteria, including decision support interventions, eCFR, HHS. Accessed September 28, 2026.
- 45 CFR 164.504, Organizational requirements for uses and disclosures, eCFR, HHS. Accessed September 28, 2026.
- 45 CFR 164.502, Uses and disclosures of protected health information, eCFR, HHS. Accessed September 28, 2026.
- AI Risk Management Framework, NIST. Accessed September 29, 2026.