Implementation of machine learning applications in healthcare organizations: A systematic review of empirical studies

A growing enthusiasm for machine learning (ML) has been noted among academics and healthcare practitioners. Despite the transformative potential of ML-based applications for patient care, their uptake and implementation in healthcare organizations is still sporadic. Numerous challenges currently impede or delay widespread implementation of ML in clinical practice, and limited knowledge is available regarding how these challenges have been addressed so far. The aim of this work is twofold: i) to examine the characteristics of the ML-based applications and the implementation process in clinical practice, using the Consolidated Framework for Implementation Research (CFIR) as theoretical guidance; ii) to synthesize the strategies adopted by healthcare organizations to foster successful implementation of ML. A systematic literature review was conducted based on the PRISMA guidelines. The search was conducted using three databases (PubMed, Scopus, and Web of Science) over a 10-year time frame (2013-2023). The search strategy was built around four blocks of keywords (artificial intelligence, implementation, health care, and study type). Only empirical studies documenting the implementation of ML applications in clinical settings were considered. The implementation process was investigated using a thematic analysis and coding procedure. The study protocol was registered in PROSPERO with registration number 403873. Thirty-four studies were selected for data synthesis. Selected papers were relatively recent, with only 9% of the records published before 2019. ML-based applications were implemented mostly within hospitals (29/34, 85%). In terms of clinical workflow, ML-based applications supported mostly prognosis and diagnosis, as observed in 20 (59%) and 10 (29%) studies, respectively. The implementation efforts were analyzed using the CFIR domains. As for the inner setting, access to knowledge and information (12/34, 35%), IT infrastructure (11/34, 32%), and organizational culture (9/34, 26%) were amongst the most observed dimensions influencing the success of implementation. As for the ML innovation itself, factors deemed relevant were its design (15/34, 41%), relative advantage with respect to existing clinical practice (14/34, 41%), and perceived complexity (14/34, 41%). As for the other domains – i.e., processes, roles, and outer setting, stakeholder engagement (12/34, 35%), reflecting and evaluating practices (11/34, 32%) and the presence of implementation leaders (9/34, 26%) were the main factors identified as salient. This study contributes to shed some light on the factors that are relevant and that should be accounted for in an implementation process of ML-based applications in healthcare. While the relevance of ML-specific dimensions, like trust, emerges clearly across several implementation domains, the evidence from this study highlighted that relevant implementation factors are not necessarily specific for ML, but rather transversal for digital health technologies. More research is needed to further clarify the factors that are relevant to implementing ML-based applications at the organizational level, and to support their uptake within healthcare organizations.