Regulation of clinical Artificial Intelligence (AI) in the Age of Agents: Unconfined Non-Deterministic Clinical Software (UNDCS) systems for healthcare
Abstract
In a recent article, Weissman et al.1 examined the extent to which artificial intelligence (AI)-based large language models (LLMs) generate clinical decision support (CDS) outputs that meet the criteria of regulated medical devices1 and called for new regulations for LLM-based CDS systems. In this manuscript, we respond to the proposed considerations highlighting those that have been addressed by existing guidelines and would not need new frameworks, as well as the need for new regulations of "generalized" CDSS that are not anchored to specific clinical indications. We contextualise this regulatory gap with an overview of the literature distinguishing between confined and unconfined AI systems. We also outline specific areas in which new regulations may be required, along with risk mitigation strategies that could be incorporated in new guidelines.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.