Diving brief:
- Healthcare professionals are using artificial intelligence tools that have not been approved by their organization, posing a potential risk to patient safety and data privacy, according to a survey released Thursday by Wolters Kluwer Health.
- More than 40% of medical workers and administrators said they knew their colleagues were using “shadow AI” products, while nearly 20% said they had used an unauthorized AI tool themselves, according to the survey by the information and software services company.
- These unapproved tools could be useful to individual workers, but their health systems have not reviewed product risks or considered governance processes, according to Dr. Peter Bonis, chief medical officer at Wolters Kluwer. “The question is, how safe are they? How effective are they and what are the risks associated with them?” he said. “And are these sufficiently recognized by the users themselves?
Dive overview:
Shadow AI, when employees use AI products that have not been authorized by their company, can be a problem. serious safety risk across all sectors, experts say. Due to the stealthy nature of shadow AI, organizational leaders (and IT teams) do not have insight into how the tools are used, creating opportunities for cyber attacks and attacks. data breaches.
Cybersecurity is already a challenge for healthcare organizations, as cybercriminals frequently target the sector due to its valuable data stores and the high stakes of care delivery.
The risks may be more serious in health care, Bonis said. Accuracy is a concern because AI tools can give misleading or inaccurate information – potentially harmful to patients. About a quarter of providers and administrators ranked patient safety as their top concern regarding AI in healthcare, according to the survey, which included more than 500 respondents across hospitals and health systems.
“There are a whole variety of ways in which — even though the intention is for humans to be in this loop at one point or another — these tools misfire, and those misfires may not be adequately intercepted at the point of care,” Bonis said.
Still, AI has become one of the most exciting technologies for the healthcare industry, given the hope that these tools can help sort through large amounts of data, speed up administrative work, and help doctors document care and find medical information.
Some workers are turning to shadow AI tools to do this, according to the survey by Wolters Kluwer, which offers its own AI-based clinical decision support product.
More than 50% of administrators and 45% of healthcare providers reported using unauthorized products because they provided faster workflow. Additionally, nearly 40% of administrators and 27% of vendors reported using shadow AI because the tool had better features or because there were no approved products.
Finally, more than 25% of providers and 10% of administrators cited curiosity or experimentation as the reason behind using shadow AI.
Many healthcare professionals are unaware of their organization’s AI policies. According to the survey, administrators are more likely to be involved in AI policymaking than healthcare providers. But 29% of vendors say they know their organization’s key AI policies, compared to 17% of administrators.
Many providers have likely encountered some AI policies related to AI scribes — which typically record clinicians’ conversations with patients and write a clinical note — as these tools have been widely adopted across health systems, Bonis said.
“That may be why they say they are aware of it, but they may not be fully aware of everything that could be considered AI policy,” he said.