Dive Brief:

• The Food and Drug Administration is seeking feedback on how to regulate medical devices that incorporate generative artificial intelligence.
• While the FDA’s Center for Devices and Radiological Health has authorized more than 1,000 devices that use AI, most do not incorporate generative AI, which can mimic patterns to create text, images or audio. This technology holds promise for patient care, the agency said, but may introduce “unique risks” compared with traditional software and AI-enabled medical devices. 
• The CDRH published a discussion paper on Tuesday outlining a potential framework for regulating generative AI. While the paper does not represent new policy or guidance, it gives the device center the opportunity to receive feedback. The FDA is taking comments through Oct. 19.

Dive Insight:

The FDA’s device center has sought stakeholder feedback in recent years on generative AI as the agency determines its regulatory approach. The CDRH’s Digital Health Advisory Committee met for the first time in 2024 to discuss total product lifecycle considerations for generative AI-enabled devices. The committee met again last year to talk about digital mental health devices that use the technology. So far, the device center has not proposed any new policies specific to generative AI. 

One challenge is that generative AI-enabled devices have unique characteristics that make them more difficult to regulate than other types of software or AI. For example, they may accept open-ended inputs, perform multiple tasks and produce variable outputs. Because of this, the devices may also carry the risk of “hallucinations” that appear authentic, and it can also be harder to know the boundaries of a device’s intended use or track performance degradation in the real world. 

In a discussion paper, the CDRH outlines three main concepts: a risk framework for generative AI-enabled devices, a competency-based approach to premarket evaluation and ideas for postmarket monitoring. 

The risk framework considers the type of activity a device performs, with informational, non-directive action as the lowest risk type of software. For example, software that provides a risk score predicting a future cardiovascular event would fall under this category.

AI features that direct patients or clinicians to take a specific action increase in risk, with fully autonomous devices being the highest risk category. The paper acknowledged that the risk level may also depend on the circumstances. For example, a device that autonomously prescribes antibiotics for a strep throat infection may carry a different risk of harm than a device that starts thrombolytic therapy as part of a stroke workflow.

As the CDRH thinks about how to review generative AI devices before they go to market, the device center outlined a competency-based approach. Devices would be benchmarked and then tested in a clinical setting to evaluate how the final, user-facing version performs in terms of safety and effectiveness. 

Finally, the CDRH also emphasized the importance of postmarket monitoring. This would be the responsibility of device manufacturers, according to the paper, although clinicians, healthcare institutions, payers and other authorities may have a role to play. 

The device center said it is considering whether it would be appropriate to accept “greater premarket uncertainty” about the benefits and risks of a generative AI-enabled device through more reliance on postmarket monitoring.