Influence of AI-informed Disease Prevalence on Radiologist Performance: Insights from the Mammography Screening with Artificial Intelligence trial (MASAI)
Author Block: J. Gommers1, C. Abbey2, O. Hagberg3, K. Lang3; 1Nijmegen/NL, 2Santa Barbara, CA/US, 3Malmö/SE
Purpose: To investigate how knowledge of AI-informed disease prevalence influenced radiologists’ screening performance by comparing outcomes across AI risk scores when reading with and without AI.
Methods or Background: This study used data from the prospective MASAI trial, designed to compare AI-supported mammography screening, where AI triaged examinations to single (AI scores 1-9) or double (AI score 10) reading and was used as decision support, with standard double reading without AI. Recall rates (RR), cancer detection rates (CDR), and false positive rates (FPR) were compared between the AI-supported group (intervention) and double reading group (control) using Fisher’s exact tests with results stratified by AI risk scores. Single-reader assessments in the control group were approximated for examinations with AI scores 1-9.
Results or Findings: In total, 105,087 women were included in this study, of which 52,576 in the control group and 52,511 in the intervention group. For examinations classified as low risk (AI score 1–7), AI-supported reading led to a reduction in RR (0.50% vs 0.61%, P=.043) and FPR (0.49% vs 0.59%, P=.049 ), without affecting CDR (0.17‰ vs 0.22‰, P=.804), compared to standard reading. For intermediate-risk examinations (AI scores 8–9), AI-supported reading resulted in increased RR (2.29% vs 1.58%, P<.001) and FPR (1.90% vs 1.30%, P=.003), with no change in CDR (3.94‰ vs 2.75‰, P=.224). For high-suspicion examinations (AI score 10), AI-supported reading increased RR (14.41% vs 9.44%, P<.001), FPR (6.69% vs 3.40%, P<.001), and CDR (77.23‰ vs 60.34‰, P=.004).
Conclusion: Knowledge about AI-informed prevalence affects radiologists differently across AI risk categories, enhancing cancer detection in high-risk examinations and reducing false positives in low-risk examinations.
Limitations: Interval cancer data not yet included.
Funding for this study: Swedish Cancer Society, Confederation of Regional Cancer Centres, and Swedish governmental funding for clinical research.
Has your study been approved by an ethics committee? Yes
Ethics committee - additional information: The Swedish Ethical Review Authority approved the study (2020-04936, 2023-026848-02) and waived the need for obtaining written informed consent.