FDA Submission Recommendations for AI-Enabled Device Software Functions

FDA released a draft guidance document, “Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations” on January 7, 2025. The document provides recommendations on the contents of submissions (be it a 510(k), De Novo, PMA, HDE, or BLA) to FDA for devices that include AI-enabled device software functions and provides recommendations for the design and development of AI-enabled devices that manufacturers may consider using throughout the total product lifecycle (TPLC). In the document, an AI-enabled device is a device that includes one or more AI-enabled device software functions (AI-DSFs).

An AI-DSF is a device software function that implements one or more AI models to achieve its intended purpose (where a model is a mathematical construct that generates an inference or prediction based on new input data).

The guidance is full of useful information for sponsors of AI-enabled devices. For example, the guidance notes that for a 510k, an AI-enabled device can be found substantially equivalent to a non-AI-enabled device with the same intended use provided, among other things, the AI-enabled device does not introduce different questions of safety and effectiveness compared to the non-AI-enabled device and meets other requirements for a determination of substantial equivalence. The guidance is quite long – 64 pages! Most of the information in the guidance is content recommendations for a marketing submission of an AI-enabled device. Below are some high points of each section of the content recommendations.

Device Description

In the “Device Description” portion of a marketing submission, sponsors should provide a device description to help FDA understand the general characteristics of the AI-enabled device. Sponsors should include:

  • Statement that AI is used in the device
  • Description of the device inputs & outputs
  • Explanation of how AI is used to achieve the device’s intended use
  • Description of the intended users, their characteristics, and the level & type of training they are expected to have
  • Description of the intended use environment
  • Description of the intended workflow for the use of the device
  • Description of installation & maintenance procedures
  • If configurable, include: 1) a description of all configurable elements of the AI-enabled device, 2) a description of how these elements and their settings can be configured

User Interface

In the “Software Description” section of the Software Documentation portion of a marketing submission, sponsors should describe the user interface as it conveys important information about what the device is intended to do and how users are intended to interact with it. The user interface includes printed labeling (packaging and user manuals) but labeling should be submitted separately. Sponsors should include information about and descriptions of the user interface that make clear the device workflow, including the information that is provided to users, when the information is provided, and how it is presented. Possible methods to provide this type of information are:

  • Graphical representation (e.g., photographs, illustrations, wireframes, line drawings)
  • Written description of the device user interface
  • Overview of the operational sequence of the device & the user’s expected interactions with the user interface
  • Examples of the output format
  • Demonstration of the device (ex: recorded video)

Labeling

In the “Labeling” portion of a marketing submission, sponsors should address the following types of information in a format and at a reading level that is appropriate for the intended user (e.g., considering characteristics such as age, education or literacy level, sensory or physical impairments, or occupational specialty) to help ensure users can quickly access important information. Tables and graphics may be used to communicate this labeling information.

  • Statement that AI is used in the device & an explanation of how AI is used to achieve the device’s intended use
  • Description of the model inputs and instructions on any steps the user is expected to take to prepare input data for processing by the device
  • Explanation of what the model output means and how it is intended to be used
  • Explanation of the intended degree of automation the device exhibits
  • High level description of the methods and architecture used to develop the model(s) implemented in the device
  • Description of the development data including: the source(s) of data, study sites, sample size, demographic distributions, and criteria/expertise used for determining clinical reference standard
  • Description of the performance validation data including: the source(s) of data, study sites, sample size, other important study design and data structure information, primary endpoints of the validation study including pre-specified performance criteria, and
  • Criteria/expertise used for determining clinical reference standard data
  • Description of the device performance metrics, an explanation of the device performance across important subgroups, and a description of the corresponding performance for different operating points
  • Description of any methods or tools to monitor and manage device performance
  • Description of all known limitations of the AI-enabled device, AI-DSF(s), or model(s).
  • Instructions on integrating the AI-enabled device into the site’s data systems and clinical workflow
  • Description of and instructions on any customizable features
  • Explanation of any additional metrics or visualizations used to add context to the model output
  • For AI-enabled devices intended for use by patients or caregivers, manufacturers should provide labeling material that is designed for patients and caregivers describing the instructions for use, the device’s indication, intended use, risks, and limitations.

Risk Assessment

In the “Risk Management File” of the “Software Documentation” portion of a marketing submission, sponsors should include a comprehensive risk assessment to help FDA understand whether appropriate risks have been identified and how they are controlled. Sponsors should include:

  • Risk Management File composed of a risk management plan, a risk assessment, and a risk management report.
  • FDA also recommends that sponsors incorporate the considerations outlined in the FDA-recognized voluntary consensus standard of AAMI CR34971 Guidance on the Application of ISO 14971 to Artificial Intelligence and Machine Learning, which is specific to AI-enabled devices.
  • Also note that FDA recommends that consideration of risks related to understanding information (ex: information necessary to use or interpret device and/or the risk of lack of information or unclear information) should be one part of a comprehensive approach to risk management for an AI-enabled device.

Data Management

A clear explanation of the data management, including data management practices and characterization of data used in the development and validation of the AI-enabled device is critical for FDA to understand how the device was developed and validated. The data management information for data used in the development of the model should be included in the “Software Description” of the “Software Documentation” portion of the marketing submission, while the data management info for data used in the performance validation (i.e., clinical validation) documentation should be included in the “Performance Testing” portion of the marketing submission. Sponsors should include:

  • Description of data collection methods, the limitations of the dataset, quality assurance processes related to the data, the size of each data set, the mechanisms used to improve diversity in enrollment within the scope of the study, the use of synthetic data
  • Description of data cleaning/processing
  • Description of how the reference standard was established, the uncertainty inherent in the selected reference standard, protocols used if the reference standard is based on evaluations from clinicians, etc.
  • Description of the expertise of those performing the data annotation, the specific training provided to annotators to guide their annotation decisions, the methods for evaluating quality/consistency of data annotations and adjudicating disagreements, and a detailed plan for addressing incorrect data annotation
  • Description of the data storage
  • Description of the management and independence of data
  • Explanation of how the data is representative of the intended use population, a justification for any missing population characteristics, a characterization of the distribution of data along important covariates, a subgroup analysis, an explanation regarding how any OUS data compares to US population

Model Description and Development

In the “Software Description” of the “Software Documentation” portion of a marketing submission, sponsors should include a model description and model development information for each model, as this supports FDA’s ability to assess the safety and effectiveness of an AI-enabled device and determines the device’s performance testing specifications.

  • The model description provides detailed information about the technical characteristics of the model(s) themselves and the algorithms and methods that were used in their development.

Performance Validation

Validation includes ensuring that the device, as utilized by users, will perform its intended use safely and effectively, as well as establishing that the relevant performance specifications of the device can be consistently met. For AI-enabled devices, manufacturers should demonstrate users’ ability to interact with and understand the device as intended in addition to ensuring the device itself meets relevant performance specifications. This means that clinical and non-clinical testing should be included in the “Performance Testing” portion of a submission, while software verification and validation should be included in the “Software testing as part of Verification and Validation” section of the “Software Documentation” portion of a submission. To show performance validation, sponsors should include:

  • Assessment of the performance of the human-device team
  • Study protocols
  • Study results
  • Software Version History

Device Performance Monitoring

When appropriate, a device performance monitoring plan should be included in the “Risk Management File” of the “Software Documentation” portion of a marketing submission. Ongoing performance monitoring is important for AI-enabled devices because models are highly dependent on the characteristics of data used to train them, and as such, their performance can be particularly sensitive to changes in data inputs. Sponsors of AI-enabled devices that elect to employ proactive performance monitoring as a means of risk control and to provide reasonable assurance of the device’s safety and effectiveness, should include information regarding their performance monitoring plans as part of the premarket submission. A Performance Monitoring Plan may include:

  • Description of the data collection and analysis methods for identifying/assessing changes in model performance and monitoring potential causes of undesirable changes in performance
  • Description of robust software lifecycle processes that include mechanisms for monitoring in the deployment environment
  • Plan for deploying updates, mitigations, and corrective actions that address changing performance in a timely manner
  • Description of the procedures for communicating the results of performance monitoring and any mitigations to device users

Cybersecurity

In the “Cybersecurity/Interoperability” portion of a marketing submission, sponsors are recommended to include information regarding the cybersecurity controls and security risk management relevant to the AI components or features. Refer to the 2023 Premarket Cybersecurity Guidance  for more detail, but sponsors should include the following types of information:

  • Any additional elements added where there are unique considerations related to AI cybersecurity
  • Explanation of how cybersecurity testing is appropriate to address the risks associated with the model, including malformed input (fuzz) testing & penetration testing
  • Security Use Case View(s) that covers the AI-enabled considerations for the device
  • Descriptions of controls implemented to address data vulnerability and preventing data leakage

Public Submission Summary

In the “Administrative Documentation” portion of a marketing submission, if a public summary is required, sponsors should include details about the AI-enabled device to support transparency to users of FDA’s determination of substantial equivalence or reasonable assurance of safety and effectiveness for the device. Sponsors should provide the following types of information, excluding any patient identifiers, trade secrets, and confidential commercial information:

  • Statement that AI is used in the device
  • Explanation of how AI is used as part of the device’s intended use
  • Description of the class of model and limitations of the model within the device description
  • Description of the development and validation datasets and information about the demographic characteristics in the population(s) of intended use
  • Description of the statistical confidence level of predictions
  • Description of how the model will be updated and maintained over time

 

The guidance has six detailed appendices at the end, including an Example Model Card and an Example 510(k) Summary with Model Card.

Find more SoftwareCPR® articles on AI here: FDA Draft Guidance AI/ML or here: Retaining Training Data Sets, or click the “ai” tag on the article.

About the author

Amy enjoys researching and writing about developments in medical technology and how that intersects with US law. She received her J.D. from the University of Florida Levin College of Law in 2020 and now works as a Regulatory Associate for SoftwareCPR®, a general-purpose regulatory consulting firm that is recognized globally for their expertise with standards and national regulations pertaining to medical device, mobile medical app, and HealthIT software.

SoftwareCPR Training Courses:

IEC 62304 and other emerging standards for Medical Device and HealthIT Software

Our flagship course for preparing regulatory, quality, engineering, operations, and others for the activities and documentation expected for IEC 62304 conformance and for FDA expectations. The goal is to educate on the intent and purpose so that the participants are able to make informed decisions in the future.  Focus is not simply what the standard says, but what is meant and discuss examples and approaches one might implement to comply.  Special deep discount pricing available to FDA attendees and other regulators.

3-days onsite with group exercises, quizzes, examples, Q&A.

Instructor: Brian Pate

Next public offering:  June 24-26, 2025 (Boston, MA)

Multiple participants from the same company: If you register 5 or more from the same company before March 15, 2025, receive a special discounted registration of $1999 per person.  These registrations may be transferred to another person at any time. Email training@softwarecpr.com to register and secure the TEAM discount.

 

For private, in-house courses, please contact us.

Email training@softwarecpr.com for more info.

 


 

Being Agile & Yet Compliant (Public)

Our SoftwareCPR unique approach to incorporating agile and lean engineering to your medical device software process training course is now open for registration!

  • Agile principles that align well with medical
  • Backlog management
  • Agile risk management
  • Incremental and iterative software development lifecycle management
  •  Frequent release management
  • And more!

3 days virtual (Zoom) with group exercises, quizzes, examples, Q&A.

Lead Instructor: Mike Russell

Next public offerings:

  • Americas: 11-13 February 2025
  • EU/Eastern Europe/Middle East/Africa/Atlantic/eastern South America: 18-20 February 2025
  • Southern Central Northeastern Pacific: 24-26 February 2025
See our post titled: 1st Quarter 2025 Agile Compliant Courses Scheduled

 

Medical Device Cybersecurity (Public or Private)

This course takes a deep dive into the US FDA expectations for cybersecurity activities in the product development process with central focus on the cybersecurity risk analysis process. Overall approach will be tied to relevant standards and FDA guidance documentation. The course will follow the ISO 14971:2019 framework for overall structure but utilize IEC 62304, IEC 81001-5-1, and AAMI TIR57 for specific details regarding cybersecurity planning, risk characterization, threat modeling, and control strategies.

2-days onsite with group exercises, quizzes, examples, Q&A.

Instructor: Dr Peter Rech, 2nd instructor (optional)

Next public offering:  TBD

Corporate Office

15148 Springview St.
Tampa, FL 33624
USA
+1-781-721-2921
Partners located in the US (CA, FL, MA, MN, TX) and Canada.