Effect of an integrated CT-based lung cancer detection and diagnosis AI tool on radiologists evaluating lung nodules: A multi-reader multi-case study
Author Block: M. Santos1, C. Santos1, J. H. R. Cairns2, M. Darby2, A. Johnstone2, C. Arteta1, A. Scarsbrook2; 1Oxford/UK, 2Leeds/UK
Purpose: Integration of AI-based computer-aided detection and diagnosis tools into lung cancer screening has the potential to improve and standardize CT reporting, streamline follow-up recommendations, reduce diagnostic errors, and increase efficiency. This multi-reader, multi-case (MRMC) study evaluated the impact of a new AI tool, assessing influence on risk stratification of pulmonary nodules by radiologists.
Methods or Background: A fully crossed MRMC design involved twelve radiologists, with varying experience and sub-speciality expertise, retrospectively reviewing 240 screening and non-screening thoracic CTs (95 lung cancers), with and without AI support. AI assistance consisted of automated localisation, measurement, and characterisation of detected lung nodules, including a per-nodule lung cancer risk score. A 30-day washout period separated the two reads of any given case. Sequencing was randomised with AI-assistance occurring either during the first or second read.
Percentage likelihood of malignancy was estimated by the reader or AI tool. Performance of AI-assisted versus unassisted read against ground truth was compared using area under the curve (AUC) analysis, averaged across readers. Statistical significance of mean AUC difference was performed using Dorfman-Berbaum-Metz methodology.
Results or Findings: Mean effect size between assisted and unassisted reads was 3.92%, 95% confidence interval (CI) [2.00, 5.85] (p < 0.001). When stratified by reader subspeciality, mean effect size for cardiothoracic radiologists (n=7) was lower (2.54%, [0.75, 4.34], p=0.009) compared to other subspecialties (5.86%, [3.52, 8.2], p<0.001). Similarly, when comparing experienced (n=4) versus less experienced participants, mean effect size was lower 2.62%, [-0.16, 5.4], p=0.06 and 4.58%, [2.25, 6.9], p<0.001, respectively.
Conclusion: The study illustrates the potential utility of an integrated detection and diagnosis AI tool to support lung cancer screening CT reporting, with higher impact for less experienced and non-specialist radiologists.
Limitations: Provisional evaluation with 12 participants in the MRMC study.
Funding for this study: The study was jointly funded by the National Institute for Health and Care Research (NIHR) and the Office for Life Sciences (OLS) under project ID NIHR207547.
Has your study been approved by an ethics committee? Not applicable
Ethics committee - additional information: