FDA released a draft guidance document, “Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations” on January 7, 2025. The document provides recommendations on the contents of submissions (be it a 510(k), De Novo, PMA, HDE, or BLA) to FDA for devices that include AI-enabled device software functions and provides recommendations for the design and development of AI-enabled devices that manufacturers may consider using throughout the total product lifecycle (TPLC). In the document, an AI-enabled device is a device that includes one or more AI-enabled device software functions (AI-DSFs).
An AI-DSF is a device software function that implements one or more AI models to achieve its intended purpose (where a model is a mathematical construct that generates an inference or prediction based on new input data).
The guidance is full of useful information for sponsors of AI-enabled devices. For example, the guidance notes that for a 510k, an AI-enabled device can be found substantially equivalent to a non-AI-enabled device with the same intended use provided, among other things, the AI-enabled device does not introduce different questions of safety and effectiveness compared to the non-AI-enabled device and meets other requirements for a determination of substantial equivalence. The guidance is quite long – 64 pages! Most of the information in the guidance is content recommendations for a marketing submission of an AI-enabled device. Below are some high points of each section of the content recommendations.
Device Description
In the “Device Description” portion of a marketing submission, sponsors should provide a device description to help FDA understand the general characteristics of the AI-enabled device. Sponsors should include:
- Statement that AI is used in the device
- Description of the device inputs & outputs
- Explanation of how AI is used to achieve the device’s intended use
- Description of the intended users, their characteristics, and the level & type of training they are expected to have
- Description of the intended use environment
- Description of the intended workflow for the use of the device
- Description of installation & maintenance procedures
- If configurable, include: 1) a description of all configurable elements of the AI-enabled device, 2) a description of how these elements and their settings can be configured
User Interface
In the “Software Description” section of the Software Documentation portion of a marketing submission, sponsors should describe the user interface as it conveys important information about what the device is intended to do and how users are intended to interact with it. The user interface includes printed labeling (packaging and user manuals) but labeling should be submitted separately. Sponsors should include information about and descriptions of the user interface that make clear the device workflow, including the information that is provided to users, when the information is provided, and how it is presented. Possible methods to provide this type of information are:
- Graphical representation (e.g., photographs, illustrations, wireframes, line drawings)
- Written description of the device user interface
- Overview of the operational sequence of the device & the user’s expected interactions with the user interface
- Examples of the output format
- Demonstration of the device (ex: recorded video)
Labeling
In the “Labeling” portion of a marketing submission, sponsors should address the following types of information in a format and at a reading level that is appropriate for the intended user (e.g., considering characteristics such as age, education or literacy level, sensory or physical impairments, or occupational specialty) to help ensure users can quickly access important information. Tables and graphics may be used to communicate this labeling information.
- Statement that AI is used in the device & an explanation of how AI is used to achieve the device’s intended use
- Description of the model inputs and instructions on any steps the user is expected to take to prepare input data for processing by the device
- Explanation of what the model output means and how it is intended to be used
- Explanation of the intended degree of automation the device exhibits
- High level description of the methods and architecture used to develop the model(s) implemented in the device
- Description of the development data including: the source(s) of data, study sites, sample size, demographic distributions, and criteria/expertise used for determining clinical reference standard
- Description of the performance validation data including: the source(s) of data, study sites, sample size, other important study design and data structure information, primary endpoints of the validation study including pre-specified performance criteria, and
- Criteria/expertise used for determining clinical reference standard data
- Description of the device performance metrics, an explanation of the device performance across important subgroups, and a description of the corresponding performance for different operating points
- Description of any methods or tools to monitor and manage device performance
- Description of all known limitations of the AI-enabled device, AI-DSF(s), or model(s).
- Instructions on integrating the AI-enabled device into the site’s data systems and clinical workflow
- Description of and instructions on any customizable features
- Explanation of any additional metrics or visualizations used to add context to the model output
- For AI-enabled devices intended for use by patients or caregivers, manufacturers should provide labeling material that is designed for patients and caregivers describing the instructions for use, the device’s indication, intended use, risks, and limitations.
Risk Assessment
In the “Risk Management File” of the “Software Documentation” portion of a marketing submission, sponsors should include a comprehensive risk assessment to help FDA understand whether appropriate risks have been identified and how they are controlled. Sponsors should include:
- Risk Management File composed of a risk management plan, a risk assessment, and a risk management report.
- FDA also recommends that sponsors incorporate the considerations outlined in the FDA-recognized voluntary consensus standard of AAMI CR34971 Guidance on the Application of ISO 14971 to Artificial Intelligence and Machine Learning, which is specific to AI-enabled devices.
- Also note that FDA recommends that consideration of risks related to understanding information (ex: information necessary to use or interpret device and/or the risk of lack of information or unclear information) should be one part of a comprehensive approach to risk management for an AI-enabled device.
Data Management
A clear explanation of the data management, including data management practices and characterization of data used in the development and validation of the AI-enabled device is critical for FDA to understand how the device was developed and validated. The data management information for data used in the development of the model should be included in the “Software Description” of the “Software Documentation” portion of the marketing submission, while the data management info for data used in the performance validation (i.e., clinical validation) documentation should be included in the “Performance Testing” portion of the marketing submission. Sponsors should include:
- Description of data collection methods, the limitations of the dataset, quality assurance processes related to the data, the size of each data set, the mechanisms used to improve diversity in enrollment within the scope of the study, the use of synthetic data
- Description of data cleaning/processing
- Description of how the reference standard was established, the uncertainty inherent in the selected reference standard, protocols used if the reference standard is based on evaluations from clinicians, etc.
- Description of the expertise of those performing the data annotation, the specific training provided to annotators to guide their annotation decisions, the methods for evaluating quality/consistency of data annotations and adjudicating disagreements, and a detailed plan for addressing incorrect data annotation
- Description of the data storage
- Description of the management and independence of data
- Explanation of how the data is representative of the intended use population, a justification for any missing population characteristics, a characterization of the distribution of data along important covariates, a subgroup analysis, an explanation regarding how any OUS data compares to US population
Model Description and Development
In the “Software Description” of the “Software Documentation” portion of a marketing submission, sponsors should include a model description and model development information for each model, as this supports FDA’s ability to assess the safety and effectiveness of an AI-enabled device and determines the device’s performance testing specifications.
- The model description provides detailed information about the technical characteristics of the model(s) themselves and the algorithms and methods that were used in their development.
Performance Validation
Validation includes ensuring that the device, as utilized by users, will perform its intended use safely and effectively, as well as establishing that the relevant performance specifications of the device can be consistently met. For AI-enabled devices, manufacturers should demonstrate users’ ability to interact with and understand the device as intended in addition to ensuring the device itself meets relevant performance specifications. This means that clinical and non-clinical testing should be included in the “Performance Testing” portion of a submission, while software verification and validation should be included in the “Software testing as part of Verification and Validation” section of the “Software Documentation” portion of a submission. To show performance validation, sponsors should include:
- Assessment of the performance of the human-device team
- Study protocols
- Study results
- Software Version History
Device Performance Monitoring
When appropriate, a device performance monitoring plan should be included in the “Risk Management File” of the “Software Documentation” portion of a marketing submission. Ongoing performance monitoring is important for AI-enabled devices because models are highly dependent on the characteristics of data used to train them, and as such, their performance can be particularly sensitive to changes in data inputs. Sponsors of AI-enabled devices that elect to employ proactive performance monitoring as a means of risk control and to provide reasonable assurance of the device’s safety and effectiveness, should include information regarding their performance monitoring plans as part of the premarket submission. A Performance Monitoring Plan may include:
- Description of the data collection and analysis methods for identifying/assessing changes in model performance and monitoring potential causes of undesirable changes in performance
- Description of robust software lifecycle processes that include mechanisms for monitoring in the deployment environment
- Plan for deploying updates, mitigations, and corrective actions that address changing performance in a timely manner
- Description of the procedures for communicating the results of performance monitoring and any mitigations to device users
Cybersecurity
In the “Cybersecurity/Interoperability” portion of a marketing submission, sponsors are recommended to include information regarding the cybersecurity controls and security risk management relevant to the AI components or features. Refer to the 2023 Premarket Cybersecurity Guidance for more detail, but sponsors should include the following types of information:
- Any additional elements added where there are unique considerations related to AI cybersecurity
- Explanation of how cybersecurity testing is appropriate to address the risks associated with the model, including malformed input (fuzz) testing & penetration testing
- Security Use Case View(s) that covers the AI-enabled considerations for the device
- Descriptions of controls implemented to address data vulnerability and preventing data leakage
Public Submission Summary
In the “Administrative Documentation” portion of a marketing submission, if a public summary is required, sponsors should include details about the AI-enabled device to support transparency to users of FDA’s determination of substantial equivalence or reasonable assurance of safety and effectiveness for the device. Sponsors should provide the following types of information, excluding any patient identifiers, trade secrets, and confidential commercial information:
- Statement that AI is used in the device
- Explanation of how AI is used as part of the device’s intended use
- Description of the class of model and limitations of the model within the device description
- Description of the development and validation datasets and information about the demographic characteristics in the population(s) of intended use
- Description of the statistical confidence level of predictions
- Description of how the model will be updated and maintained over time
The guidance has six detailed appendices at the end, including an Example Model Card and an Example 510(k) Summary with Model Card.
Find more SoftwareCPR® articles on AI here: FDA Draft Guidance AI/ML or here: Retaining Training Data Sets, or click the “ai” tag on the article.
