AI failures during AI implementation into clinical practice
Author Block: N. Stogiannos1, R. Cuocolo2, A. D'Antonoli3, D. Pinto Dos Santos4, H. Harvey1, M. Huisman5, B. Koçak6, M. Klontzas7, C. Malamateniou1; 1London/UK, 2Naples/IT, 3Basel/CH, 4Mainz/DE, 5Nijmegen/NL, 6Instanbul/TR, 7Heraklion/GR
Purpose: To map out different potential causes of AI failures, that could potentially impede AI implementation, lead to poor patient outcomes, increase financial costs, and add burden to clinical workflows. Potential solutions to mitigate these errors are also presented.
Methods or Background: A diverse group of AI experts in medical imaging, including radiologists, radiographers, computer scientists, and technical physicians, hand-searched all available literature to identify studies related to AI failures in radiology, as well as potential solutions. All eligible papers were then analysed by three group members and assigned to specific categories.
Results or Findings: Three distinct categories of AI failures were identified: a) errors related to AI models (algorithmic bias, lack of diverse and inclusive datasets, poor internal/external testing, failures due to unseen real-world data, suboptimal post-market surveillance, and data safety during AI model decommissioning), b) infrastructure-related failures (hardware/software issues, poor integration into PACS/RIS, network deficiencies), and c) human factors (human-AI interaction, automation bias, resistance to change, publication mishaps, annotation/interpretation errors, ergonomics). Adequate AI training and literacy, continuous monitoring of AI tools, standardized reporting of AI studies, multidisciplinary collaboration, effective leadership, and funding were suggested as potential solutions to the above failures.
Conclusion: AI models can fail at any stage through their lifecycle. Infrastructure issues and human related factors may present an important cause of AI failures.
Limitations: This is not an exhaustive list of AI failures in radiology as it is challenging to identify AI failures in a culture that mostly celebrates success. Also, identification is challenging due to publication bias and the evolving nature of the field.
Funding for this study: N/A
Has your study been approved by an ethics committee? Not applicable
Ethics committee - additional information: