Multimodal AI model outperforms established biomarkers for immunotherapy in lung cancer

The international I3LUNG study, enrolling 2,396 patients across six countries, demonstrates that artificial intelligence models utilizing routine clinical and blood data outperform traditional biomarkers like PD-L1 in predicting lung cancer immunotherapy outcomes. While multimodal integration of imaging and genomics shows promise in training, its real-world clinical benefit remains unconfirmed. However, a clinical usability study reveals that pairing physicians with the explainable AI tool significantly improves treatment response prediction accuracy.

I3LUNG multimodal AI study

• ▪The international, multicenter I3LUNG study (NCT05537922), funded under the Horizon Europe 2021–2027 framework, enrolled 2,396 patients to develop artificial intelligence-based decision support systems for non-small cell lung cancer immunotherapy
• ▪The retrospective arm of the I3LUNG study analyzed data from 2,396 patients with stage IIIC–IVB advanced non-small cell lung cancer treated with immunotherapy between September 2012 and October 2023
• ▪The I3LUNG study collected patient data across six clinical centers located in Italy, Greece, Germany, Spain, the United States, and Israel
• ▪The I3LUNG study integrated four distinct patient data modalities: clinical and blood data, computed tomography scans, digital pathology slides, and genomic analyses

AI outperforms PD-L1 biomarkers

• ▪Artificial intelligence models developed in the I3LUNG study significantly outperformed the clinically approved programmed death ligand 1 biomarker in predicting non-small cell lung cancer immunotherapy outcomes
• ▪The I3LUNG artificial intelligence models outperformed established clinical scores and biomarkers, including the Eastern Cooperative Oncology Group performance status, neutrophil-to-lymphocyte ratio, lactate dehydrogenase, and the Lung Immune Prognostic Index

Blood-only model results

• ▪Post hoc global explainability analysis of the I3LUNG clinical and blood-only models identified Eastern Cooperative Oncology Group performance status greater than 0 and bone, liver, and brain metastases as strong negative prognostic factors
• ▪The I3LUNG machine learning and deep learning clinical and blood-only models achieved area under the curve values up to 0.77 in the independent test set
• ▪The I3LUNG clinical and blood-only models showed a performance drop in the external validation cohort from the University of Chicago, with area under the curve values ranging from 0.55 to 0.72

Multimodal integration challenges

• ▪Deep learning intermediate fusion models developed in the I3LUNG study did not show added predictive benefit from integrating multiple modalities compared to clinical and blood-only models
• ▪Although multimodal integration of clinical, blood, pathology, and radiology data showed higher performance in cross-validation, its incremental benefit did not consistently translate to the independent test and external validation sets
• ▪The retrospective multimodal analysis in the I3LUNG study was limited by data sparsity, with only 339 of the 2,075 patients in the main cohort having complete data across all four modalities

Physician decision support tools

• ▪The agreement on disease control rate predictions between expert and nonexpert physicians improved from slight (kappa of 0.11) to moderate (kappa of 0.48) when using the I3LUNG artificial intelligence tool
• ▪The use of the I3LUNG explainable artificial intelligence tool improved the participating physicians' disease control rate prediction sensitivity from 0.72 to 0.87 and overall accuracy from 0.57 to 0.65
• ▪A clinical usability study involving 20 physicians demonstrated that both lung cancer experts and nonexperts improved their treatment outcome predictions when using the I3LUNG explainable artificial intelligence decision support tool

Debatable claims

• ▪AI models should replace PD-L1 as the primary biomarker for lung cancer immunotherapy
• ▪Oncologists should rely on AI decision support tools to guide immunotherapy selection