Medical device industry continues to turn to AI

For nearly 13 years, countries have collaborated on medical device guidance to develop a common set of standards for regulators around the world. The work has been part of the International Medical Device Regulators Forum (IMDRF), which will meet next week in Washington, D.C.

Members include the European Union, Japan, Canada, Australia and the U.S., and the World Health Organization is an official observer. The group has developed guidance documents and discussed topics ranging from early questions such as what is a medical device to how countries can regulate new technologies like artificial intelligence.

Melissa Torres, the Center for Devices and Radiological Health’s associate director of international affairs, spoke to MedTech Dive about the IMDRF’s push to get more countries involved in the group, helping countries with newer device regulatory systems develop, and the challenges around overseeing fast-moving technologies like AI and software as a medical device.

This interview has been edited for length and clarity.

MEDTECH DIVE: What is a top priority or topic that IMDRF will focus on either at the meeting next week or throughout 2024?

MELISSA TORRES: As soon as the FDA took over chairmanship in 2024, one of the big areas that we wanted to prioritize was increasing engagement within IMDRF. Last year, we created a new membership category called the “affiliate membership category.” Not-so-well-developed regulatory authorities — for example, a regulator that’s thinking about developing a medical device regulatory system — can apply to IMDRF and be part of the group. They can engage and understand what IMDRF is doing. Because many of these countries are thinking, “How do we implement some of these documents in our jurisdiction?” IMDRF has documents as simple as the classification of a medical device or the definition of a specific medical device. All of that is encompassed within a lot of the IMDRF documents.

We wanted to think about a mechanism for these countries to be able to engage and understand what's going on within the group. This new membership category gives them an opportunity to do so. Before this category, it was really difficult to become a member. You had to have a fully developed regulatory system, you had to be able to do pre-market and post-market reviews. It didn’t offer the opportunity for these countries to engage. We’re trying to continue what the European Union [IMDRF’s 2023 chair] had started and increase engagement and membership.

It's important for these countries to not reinvent the wheel. They may have three people at their regulatory authority, and they think they could do it all. We tell them to think about ways they can rely on these other regulators that are doing premarket reviews, for example, and how can they rely on our decisions already and take that at face value.

What is IMDRF’s AI working group focused on, and what is IMDRF looking at regarding the proliferation of AI in healthcare?

We’ve recently done some multilateral work with the U.K. and Health Canada on good machine learning practices (GMLPs). We issued our first document on GMLPs, and we’ve done predetermined change control plans as well. 

IMDRF is building upon that work. The first step was looking at GMLPs. We've taken that work — those basic principles we identified — and brought that work into IMDRF to create a more detailed document with the organization. That's what the AI working group is currently working on.

In January, FDA Commissioner Robert Califf spoke about the challenges around AI, such as the complexity of oversight and the lack of manpower as the use of the technology skyrockets. Do other countries have similar challenges?

It's all of it, right? I think we're all struggling on how to fit it into the regulatory framework. It's so fast evolving. We've heard that same challenge from other regulators that are at the table. Even countries that are just developing these regulatory systems ask, “How are you doing this? How are you handling AI?” They don’t even know how to classify medical devices yet, and they’re struggling with what to do about AI. It’s certainly a challenging topic.

I was just in Korea. We had a joint workshop we did between the FDA and Korea’s Ministry of Food and Drug Safety, the country’s medical device regulator, and it was all focused on AI and medical product development. The workshop not only focused on devices, but it also talked about drugs and biologics — we kind of covered the gamut. But it was the same messaging we heard across the board from many regulators: the challenge of manpower, and how to fit AI into regulatory systems that have existed for X number of years and are not really accounting for the fast-changing and evolving technologies right now that we're dealing with.

Is IMDRF interacting with groups like Advamed or Medtech Europe as it looks at AI, for example, to hear how companies are thinking about using the technology and developing it into devices?

Some of the working groups are only open to regulators, and then other groups are for regulators and industry. AI is actually one of the groups where we have key stakeholders plus the regulators involved because of the vast nature of that topic. We need to have their feedback in the process as well.

What are the biggest challenges IMDRF faces going forward?

I think it is in these areas, AI and software as a medical device. Some of the same challenges we were just talking about. How do you create harmonized documents internationally in these topic areas that are so fast evolving? Because while we're creating concepts that are sort of at a higher level, they still become outdated so quickly. How do we keep up on this? And how do we think about it continuously? It is a challenge — everybody's facing that same challenge — but it does give a great opportunity for us to be able to discuss it with all the right stakeholders at the table. I do see these as being the two areas that are going to be the most challenging as we move forward.

Dive Brief:

• Artificial intelligence will help medical device companies expand their portfolios and revenue streams while gaining efficiencies and curbing costs, Moody's Ratings predicts in a new report.
• The report, which Moody's published Monday, anticipates AI will start to have a positive impact on medical device companies in the next two years. Large companies are best placed to harness AI, the report states, but all companies will need to invest.
• Moody's sees data management and cybersecurity risk as key credit challenges, noting that companies may need to collect and protect large amounts of sensitive data to succeed in AI.

Dive Insight:

Moody's, a provider of credit ratings, research and risk analysis, created the report to help understand how AI may affect the credit quality of companies in the medical device, pharmaceutical and healthcare services industries through 2030. In trying to answer that question, the report provides an overview of the ways AI could affect the medical device industry in the coming years.

The analysts see AI supporting improved product offerings at medical device companies and cites examples of how companies are already using AI. The report lists Smith & Nephew’s AI-based surgical software and Zimmer Biomet’s work on smart knee implants as examples of existing projects. AI submissions to the Food and Drug Administration have surged in recent years. 

With Moody’s also expecting AI to improve productivity and efficiency, the analysts see AI having a positive impact on the credit quality of medical device companies in the next two years and continuing to bring benefits from 2026 to 2030.

The benefits may be unevenly distributed. Moody’s sees larger companies as better positioned to use AI because they have more money, personnel and proprietary data. Access to proprietary data could allow companies to customize AI products and services so they sustain a competitive advantage even as the technologies become more widely available and commoditized, the report said.

Companies will face risks as they work to realize the benefits. The large volume of sensitive data used in healthcare AI “brings risks related to data quality and an increased threat of cyberattacks,” the analysts said. Inaccurate or incomplete data can result in flawed AI outputs and decisions, the report states, and the data are valuable to hackers. The analysts see smaller companies as more vulnerable to cyberattacks.

The report also looks at the effect of regulation on AI. Moody’s expects regulation to play “a fundamental role in the ultimate credit impact of AI.” However, the impact is uncertain because rules on AI use are still in their early stages and differ by region.

New legislation could force companies to spend more on compliance. The FDA has plans for the regulation of AI. In the European Union, lawmakers have now agreed on a version of the AI Act that they will put to a vote this year. Moody’s said it believes the law could limit AI's full benefits for credit quality, depending on how it is implemented.

Sponsored content

By JLL

What if medtech companies could harness the power of AI to streamline workflows, attract top talent and deliver high-performance workplaces? Fueled by the power of artificial intelligence (AI), this future is not just a possibility–it is a reality.  

The medtech industry is on the cusp of a transformative era. The convergence of key enabling technologies–AI, Robotics and advanced materials science–is reshaping the art of the possible across work, workforce and workplace. Take for example software development. Much of medtech product innovation and development workflows are built-on software. It is no surprise that cracking the code of software development remains a top priority. Generative AI tools like GitHub Copilot X can automate coding tasks, generate test cases and document code, allowing software development to be completed up to 55% faster. Think about how progress like this will speed up software development for medical devices and healthcare applications.  

AI can also aid in discovering new product designs, identifying potential flaws faster in manufacturing and analyzing patient data to predict the likelihood of diseases for early detection and intervention. For example, Edwards Lifesciences is using AI capabilities with their new Acumen heart catheters to predict potential complications during heart surgery. This AI system analyzes real-time data and alerts surgeons to potential issues, enabling them to take proactive measures and potentially save lives. 

Further, companies are using the power of AI to equip their people. Talent continues to be a top priority for medtech companies–an AI-enabled workforce could be a true differentiator. Companies are already racing ahead with implementation to gain a competitive advantage. For example, Moderna unlocked "thousands of hours of productivity" through its AI platform, mChat, launched in partnership with OpenAI. AI-powered chatbots and voice assistants can handle administrative customer requests, troubleshoot issues and determine appropriate support documentation in real-time. This allows top talent to focus on the highest ROI for time invested. 

In addition to AI's impact on work and workforce, AI-powered real estate decisions could allow medtech companies to unlock yet another frontier. Predictive models could analyze vast amounts of data related to supply chain, labor markets, transportation costs and other relevant factors to help companies identify optimal locations for their manufacturing facilities. AI algorithms could also simulate various scenarios based on different location choices, helping decision-makers understand the potential risks and benefits associated with each option. 

Attracting top talent is crucial, and locating in top life science clusters provides access to researchers, data scientists and engineers. However, differentiating in highly competitive clusters such as the San Francisco Bay Area, Minneapolis/St. Paul, Boston/Cambridge, or New York/New Jersey means companies need to do more. One of the key differentiators will be delivering experiences that create a "wow factor" for the employee. To create a frictionless and efficient work environment, leading companies are investing in smart building technologies, such as IoT sensors, digital signage and mobile apps that allow employees to easily navigate the office, book meeting rooms and access resources. The integration of AI-powered virtual assistants can further streamline tasks and support employee productivity. Importantly, this data is increasingly beneficial for executive decision-making. As an example, JLL employs a leading occupancy intelligence platform applicable to office, research and development (R&D) and other environments. Medtech leaders can have the latest occupancy and employee in-office data and leverage insights from trusted experts to make the most confident business decisions.  

The medtech industry is on the brink of a transformative era, with AI poised to revolutionize work, workforce and workplace. By leveraging AI-powered real estate strategies and responsibly integrating AI into medical workflows, medtech companies can attract top talent, enhance patient care and drive meaningful change. Collaboration between medtech companies, AI experts and healthcare professionals will be essential to ensure the successful and ethical integration of AI technologies in the industry. 

The future belongs to those who dare to reimagine the possible and harness the power of technology and space to drive meaningful change in the medtech industry. 

A growing number of medical devices include artificial intelligence or machine learning features, and the Food and Drug Administration has started tracking ones that have gone through the agency’s approval. The FDA recently updated its list of AI/ML-enabled devices, showing that nearly 700 have now received marketing authorization. 

The latest update added a total of 171 new devices, the majority of which were cleared between August 2022 and July 2023. 

Last year, when the FDA first unveiled the list, MedTech Dive found that the majority of cleared AI/ML devices were used in radiology and went through the agency’s 510(k) clearance pathway. That continued to be true, based on an analysis of the newest additions.

Here are five takeaways from the updated list: 

1. The rate of FDA submissions slowed in 2021 and 2022 but has started to pick up again

The number of AI/ML device submissions the FDA has received has increased every year since 2015. The agency saw the biggest increase between 2019 and 2020, when the number of submissions increased by 39%. The rate of submissions slowed in 2021 and 2022, but is expected to surpass 30% again this year, the agency said in a summary provided with the list.

2. GE HealthCare had the most cleared AI/ML devices

GE HealthCare and Siemens Healthineers led the list with the most cleared AI/ML devices between August 2022 and July 2023, according to an analysis by MedTech Dive. GE HealthCare has had the most authorized AI/ML devices to date. 

Other companies that topped the list include Aidoc, a Tel Aviv-based company that makes triage systems for radiologists, and Brainlab, a Munich-based firm partnering with ZimVie on surgical navigation for spine procedures. 

3. Radiology still accounts for the majority of AI/ML devices

Last year, MedTech Dive found that the majority of AI/ML devices authorized to date have been in the field of radiology. That trend has continued, with radiology accounting for 87% of new devices last year, and 79% of new devices in the first half of 2023. 

Part of the reason radiology might be so represented on the list is because of the early digitization of imaging data compared to other specialties. The 21st Century Cures Act also specifies anything that analyzes a medical image should be regulated as a medical device. The FDA sought to clarify through a final guidance last year that certain types of software should be regulated as devices, such as those that use information from electronic medical records to flag potential cases of sepsis. 

4. None of the devices on the list used generative AI

Despite rising interest in large language models, such as ChatGPT, this type of technology was not used in any of the cleared devices. As of Oct. 19, no device has been authorized that uses generative AI or artificial general intelligence or is powered by large language models, the FDA said. 

The types of models featured in the list vary in complexity. Some were “shallow,” having less than two hidden layers between the algorithm’s input and output, while others used deep learning, the FDA said. Typically, the models used a combination of approaches to achieve a result, such as using one model to generate features and another model to do classification. 

Some examples of the types of devices cleared by the FDA this year include an algorithm developed by Edwards Lifesciences to help clinicians predict the risk of a global hypoperfusion event in patients receiving advanced hemodynamic monitoring, and a radiation planning assistant developed by MD Anderson Cancer Center to help plan cancer treatment.

5. Most devices still went through the 510(k) pathway

The vast majority of AI/ML devices cleared between August 2022 and July 2023 went through the FDA’s 510(k) pathway. Just two devices received de novo clearance during that time period. This is consistent with previous years: 96% of cleared AI/ML devices went through the 510(k) pathway between 1995 and 2022, according to an analysis by MedTech Dive last year. 

Dive Brief:

• Johnson & Johnson Medtech has partnered with Nvidia to use artificial intelligence to improve surgery, the companies said Monday.
• The agreement positions J&J to use platforms from Nvidia, a maker of computer chips used for technologies like graphics and AI, to support the deployment of surgical software and devices that enable real-time data analysis in the operating room.
• J&J and Nvidia see the technologies facilitating fast, secure deployment of AI algorithms for surgical decision-making, education and collaboration.

Dive Insight:

Researchers have identified AI as a way to help surgeons navigate through the body during procedures, provide surgical robots with some level of autonomy and otherwise improve operations. Companies such as Intuitive Surgical are already using AI in their products. Medtech firms are still working to realize many of the opportunities identified by researchers. 

J&J, one of Intuitive’s competitors for the robotic surgery market, has secured access to Nvidia’s IGX and Holoscan edge computing and AI platforms. Edge computing allows data to be processed close to the user, rather than being sent to remote servers.

Moving processing to the edge eliminates the lag and security risks that can happen as data is moved to and from servers. Those benefits may be relevant to surgical uses of AI, where processing delays could negatively affect patient outcomes and the sensitivity of health information makes it important to keep the data in a secure environment. 

“Bringing advanced edge computing hardware and software to the OR enables scalability of innovation and new AI-powered solutions for clinical decision-making, education and training, and collaboration — with the ultimate goal of advancing patient care,” Shan Jegatheeswaran, J&J Medtech’s global head of digital, said in a statement. 

David Niewolny, Nvidia's director of business development for healthcare/medical, discussed potential uses for AI in surgery in a blog post about the J&J deal. The technology could support the development of an AI model that analyzes endoscope footage and identifies organs, tissue and potential tumors in real-time, Niewolny wrote in the Monday post. Algorithms could also remove personally identifiable data to allow external experts to view videos.

The Nvidia collaboration is part of J&J’s broader push to use AI. CFO Joe Wolk said on an earnings call in January the company is “looking for opportunities, whether it be aided by artificial intelligence or just infrastructure overall, as to how we can further improve the medtech profitability profile.”

Dive Brief:

• The Food and Drug Administration is seeking examples of artificial intelligence and machine learning models that can identify and predict freezing of gait events related to Parkinson’s disease. 
• Freezing of gait is a temporary loss of forward movement while walking. These episodes affect people’s quality of life and daily activities, but they can be difficult to measure because they often happen when patients are outside of a clinic or hospital setting. 
• By testing these models against its own data, the FDA hopes to better understand the ability of these technologies to provide digitally derived endpoints that could help with early disease detection and prevention or support treatment and care in the home. 

Dive Insight:

As digital health technologies proliferate, the FDA is taking a closer look at how they can capture information about a person’s health outside of a clinical setting. For Parkinson’s disease, a handful of wearable technologies have already been cleared to track symptoms. 

Apple published study results in 2021 evaluating its Motor fluctuations Monitor for Parkinson’s disease, which uses sensors to track fluctuations in resting tremor and dyskinesia. Later that year, a company called H2O Therapeutics received 510(k) clearance for its Parky app that sends Apple Watch data to a patient’s doctor. Last month, H2O received clearance for another app that provides haptic feedback when people experience freezing of gait symptoms.  

Other cleared apps that use Apple Watch data include Rune Labs’ StrivePD software and an app developed by digital health company NeuroRPM. 

Nearly 1 million people in the U.S. are estimated to have Parkinson’s disease, according to the FDA. 

“With the fast-paced innovation and proliferation of DHT capabilities, including the use of artificial intelligence, the healthcare and technology ecosystems have a responsibility to ensure and support the safety, effectiveness, and privacy of digitally-derived endpoints,” the agency said in a summary of the challenge.

To start, the agency will ask developers, innovators and researchers to submit AI models before August 2. They can use public data sources or bring their own data. 

Then, the FDA will pick the top performers from that group, and test those models against its own freezing of gait data. Its goal is to determine best practices for using AI models to develop and validate measures from wearable devices and smartphones.

The challenge is being hosted by the FDA’s Digital Health Center of Excellence, Office of Science and Engineering Laboratories and Office of Digital Transformation’s precisionFDA.

Dive Brief:

• A U.K. agency outlined its position on the regulation of artificial intelligence as a medical device in a policy paper published Tuesday.
• The Medicines and Healthcare products Regulatory Agency (MHRA) said many AI products that can be put on the market now without conformity assessment will move to a higher risk category in upcoming reforms.
• The paper explains how MHRA interprets a government AI strategy focused on principles such as safety, security and robustness and aligns it with international standards.

Dive Insight:

The U.K. government committed to five cross-sector principles for the regulation of AI in February. A consultation found “strong support” for the principles, the government said, and established them as the basis for a regulatory framework designed to keep pace with rapidly advancing AI technology. Days later, the government wrote to the MHRA to request details of its approach to AI. 

In response, the MHRA published a policy paper that describes how it is interpreting and applying the five principles. The paper covers AI as a medical device, how the MHRA is using AI internally and how the use of AI by industry may affect the agency.

AI with a medical purpose is very likely to meet the definition of a medical device, the MHRA said. Today, the agency puts many AI products in the lowest risk classification, meaning they can come to market without being assessed by a notified body. The MHRA said many AI products will move up to higher risk classifications as part of the post-Brexit medical device regulatory framework.

The agency made multiple references to the need for international alignment in its discussion of AI as a medical device, citing its use of documents from the International Organization for Standardization and the International Medical Devices Regulators Forum to mitigate the risk of bias in AI. The MHRA is also working on its own machine learning and AI development practices. 

The agency has three people, or full-time equivalents (FTEs), working on AI as a medical device activities. It plans to grow the team to 7.5 FTEs through the 2024 to 2025 financial year. The government is providing funding for two FTEs. 

The other two sections of the paper could also have implications for the medical device industry. In one section, the MHRA discusses ways it may use AI internally, such as the application of machine learning to real world data to understand links between products and clinical outcomes. The final section covers the internal use of AI by companies, which the MHRA said is generally unlikely to affect how it regulates.

When Rajiv Sethi was first trained as a surgeon, he approached it like a craft. He trained under a surgeon who taught him how to do certain procedures, and he started his own practice.

Sethi, who is executive director of the Center for Neurosciences and Spine at Virginia Mason Franciscan Health in Seattle, now is training a new generation of surgeons interested in using robotic surgery, augmented reality and predictive analytics. Not all surgeons are comfortable with this shift in culture.

“If you think about how a robot now is going to do part or most or all of your surgery, there are a lot of folks that are maybe a little bit older than me that are very, very uncomfortable with that,” Sethi said. “We’re taking the artisan out of it and we’re making the standardizable parts really the future of how physicians are trained.”

Sethi’s comments were part of a broader discussion at AdvaMed’s MedTech Conference on Monday about the use of digital technologies in healthcare and some of the barriers to adoption. These new technologies have implications not just for surgeons, but for nurses, pharmacists, paramedics and other health professionals.

“There’s this fear of losing your capability. Your identity. This is what you’ve done the last 25, 50 years, and now suddenly something else is going to do it for you — that’s the challenge,” said Bakul Patel, Google’s senior director of global digital health strategy and regulatory, and former digital health officer for the Food and Drug Administration.

Christian Eusemann, vice president of research and innovation for Siemens Healthineers, said the company sees “love yet fear from AI,” in that providers love what the technology does, but worry about the future of their jobs. One example is a feature the company implemented that automatically positions patients for CT scans to get the correct amount of radiation. 

“Initially, there was a lot of fear by the technologists, like ‘oh this is the start of AI taking away our job,’” Eusemann said. “But now, because it’s really integrated in a way that they don’t even think about it anymore, they really appreciate it. They love it.” 

No panacea

Although digital technologies promise to improve patient care and lighten clinicians' workloads, they are not a one-size-fits-all solution for the problems hospitals currently face. Developers of medical software must be wary of bias that can be introduced by data, said Robert Cohen, president of digital, robotics and enabling technologies for Stryker. 

“Some people say, ‘Hey, why don't you just get all that data from the hospital and just work with the hospital? Well, the phenotype of those patients at the hospital is not necessarily typical for all levels of income, demographics and things like that. And just inherently by going to that hospital, by getting a couple of thousand patients, you already have inherent bias,” Cohen said. 

Furthermore, getting all of that data in one place — information from surgical equipment, patient records from a hospital, doctor’s office and physical therapist — is currently very difficult. It doesn’t all exist in one place, and people in the U.S. are trying to figure out the rules of data acquisition and data use, meaning each facility can have a different policy, Cohen added. 

Technology also can’t solve all of healthcare’s labor problems, a point that was emphasized by a historic strike of Kaiser Permanente workers that paused just before the conference.  

Shawna Butler, a nurse economist and host of a podcast on nursing innovation, pointed to high vacancy rates and turnover rates at hospitals and more clinicians leaving the profession after just a few years. 

“One of the really scary stats for me that digital technology is not going to help [is] the average tenure in an ICU for nurses. It used to be 10 years … we're down to like two and a half,” Butler said. 

At major medical systems, instruments can’t be sterilized fast enough because there just aren’t enough people — a problem that technology is also unlikely to solve, Sethi said. 

“Is this going to overburden an already burdened workforce, where we don't really have an answer right now to, in my healthcare system, that 450 positions that are open, and there's just no applicants for these jobs?” Sethi said. “And these are people that are going to check you in, these are going to be people that walk you back to your colonoscopy, these are going to be people that recover you after you wake up — these jobs are not being filled. And so this is where I see the importance of empowering the workforce being a very, very key part of how we think about digital in the future.”

Dive Brief:

• The Food and Drug Administration is ramping up production of medtech guidance, adding 18 draft documents to the list of priorities for its 2024 financial year.
• In 2022 and 2023, the FDA’s A-lists featured 10 and four draft documents, respectively. The plan for 2024 is more ambitious, with the administration aiming to draft 18 documents on topics such as artificial intelligence, cybersecurity, real-world evidence and pulse oximeters. 
• Five of the priority drafts were on the FDA’s B-list last year. The B-list for 2024 is relatively short, featuring two final guidance documents and one draft text on 3D printing at the point of care.

Dive Insight:

The list of draft documents touches on many topics the agency has discussed over the past year. Lifecycle management and premarket submissions for devices enabled by AI and machine learning top the list of drafts scheduled for publication in the coming year. The FDA also plans to post drafts on predetermined change control plans, cyber devices and pulse oximeters. 

Three of the drafts cover emergency use authorizations, a process with which the FDA gained experience in the COVID-19 pandemic. The draft documents could provide a framework for future crises by setting out the FDA’s views on enforcing rules during the immediate diagnostic response to an emerging outbreak, the validation of tests and the factors that will inform its policy on unapproved tests. 

Across the five drafts promoted from the B-list, the FDA plans to provide advice on biocompatibility, the evaluation of sex- and gender-specific data in medical device clinical trials, and its conformity assessment accreditation scheme.

The FDA’s final guidance A-list features three documents. One document, on remanufacturing medical devices, is left over from last year. The other two documents cover medical device shortages and advice on predetermined change control plans for device software enabled by AI and ML.

With 21 draft and final documents on the A-list, the agency has published a short B-list of lower-priority texts. The FDA’s B-list includes final guidance on computer software assurance and voluntary malfunction summary reporting, plus a draft document on 3D printing medical devices at the point of care.

Google has been keeping busy in the healthcare industry, announcing partnerships with organizations to implement generative artificial intelligence. The search giant announced partnerships this year with health systems like Mayo Clinic to use generative AI, or algorithms that can be used to create new content like text, to improve clinician workflows.

The company is also tackling billing, with financial engagement platform Cedar announcing an agreement with Google Cloud earlier this month to use its generative AI tools to help patients understand their healthcare bills. 

Meanwhile, a Google Research tool trained specifically on medical data, called Med-PaLM 2, can pass medical license tests and may draft medical documentation in the future. 

Healthcare Dive caught up with Amy Waldron, global director for healthcare and life sciences solutions, North America, at Google Cloud, to find out what these developments mean for the future of healthcare. 

What will Google Health’s large language model (LLM) Med-PaLM mean for healthcare? Is Google concerned about stuttering uptake among providers concerning AI and LLM?

WALDRON: Med-PaLM is a large language model, but it's designed to provide high-quality answers to medical questions. Med-PaLM 2 has been trained with a medical corpus, so it's been tuned to the medical industry. To give you a perspective of the volume, there were 540 billion parameters that went into training Med-PaLM. So it's been designed to encode medical knowledge, and to answer questions and summarize insights from a variety of medical texts.   

We're definitely not in a hurry to push something like this out without the confidence of industry experts and ourselves. And yes, we are on the extreme with regards to making sure that we have ethical use of AI. We also are in tight collaboration with our customers that are testing these pilots, because there's a significant responsibility with this type of technology, especially in the clinical environment.

How will generative AI balance accuracy with risks when it comes to healthcare?   

WALDRON: All of us have to make sure this stuff is meaningful, but also safe and used responsibly. No one wants to mess this up.

I think you have to be thoughtful with what information you include with your enterprise search capability, and customers have full control over what information is leveraged. And look at how you're transforming that [search] experience by using conversational AI to be more like a conversation, versus a patient or consumer trying to look at many different sources to find the right answer.

That's an example of very low risk, high reward. The same thing when you look at when clinicians have to search through healthcare payer policies. In many cases, they're looking through tons of PDFs. Now with this technology, that enterprise search can give you the ability to search those PDFs, so when a clinician or someone on their team asks a question, that information can be served to them.

So there are quality controls. There are a lot of use cases in healthcare that make sense to start on where you can get those near-term benefits and not expose people to risk.

Google Cloud introduced its Claims Acceleration Suite in April. What feedback are you receiving from health plans, and how do you plan to grow the platform going forward? Why is prior authorization still such a big challenge for the healthcare industry and how can AI help solve it?  

WALDRON: With Claims Acceleration Suite, that was a great example of how we streamlined the prior-authorization submission, the ingestive information and the decisioning. We use AI capabilities to make that process flow.

People are testing how to use generative AI to help clinicians draft prior authorization letters. It’s that drafting we can do with generative AI that we couldn't do before.

With generative AI, creating communications that resonate with people is really important, and that's something that is going to help with payer-provider communications. But it's also going to be a tremendous asset to the communications that healthcare organizations are having with consumers.

How are health systems such as Mayo Clinic using Google Cloud to explore generative AI’s applications?   

WALDRON: There are four broad categories from a healthcare perspective where we're seeing AI and generative AI leveraged: The first one is administrative and operational efficiency. The second one is clinician and care team support. You also have patient and consumer engagement, and advancing research and development.

That clinician and care team support is where Mayo is definitely doubling down on technology. They're using our Generative AI App Builder to improve the efficiency of clinical workflows. It's a suite that has three major capabilities that healthcare customers are diving into. One is foundation models, but the other two are pretty deep: enterprise search and conversational AI. 

The head of the Food and Drug Administration has warned the U.S. needs to be “nimble in the use and regulation of large language models” to avoid being “swept up quickly by something that we hardly understand.”

FDA Commissioner Robert Califf, speaking at the 2023 Science for Patient Engagement Symposium in May, said the models are “ushering in the revolution that many of us were hoping for,” but cautioned that the sector will need to be regulated appropriately.

Califf identified bias as a specific concern, noting that the models and other algorithms “evolve after they are put into practice” and will likely need continuous adjustment throughout the life cycle to remain accurate.

Large language models (LLMs) such as ChatGPT are a type of artificial intelligence that has advanced quickly in recent years. Califf sees LLMs as having important medtech applications that improve healthcare.

“The great things are really great: imagine a world in which your questions were answered immediately in language appropriate for your literacy and numeracy; also your clinician can actually talk with you rather than spending all their time cutting, pasting and writing clinical notes; I could go on and on,” Califf said.

The commissioner added that he sees “the regulation of large language models as critical to our future.” The speech lacked details of how he envisages the U.S. regulating LLMs, beyond his comment that a “nimble” approach is needed. 

Elsewhere in the speech, Califf, the former head of medical strategy at Alphabet, discussed how digital health technologies “provide opportunities to foster more efficient conduct of clinical investigations by making the operating characteristics of specific digital measurements in their context of use.” He cited the use of digital technologies to remotely record and analyze patient data as an example of their value. 

The FDA chief added that the technologies “can help improve patient access to, and participation in, clinical investigations by potentially reducing the burden of required visits to a research site.” Califf has advanced that idea as head of the FDA by releasing draft guidance on decentralized clinical trials.