Article Text
Abstract
Background Despite the clinical success of immune checkpoint blockade therapies, many patients do not respond to treatments or become resistant. Previous attempts to predict treatment efficacy suffered from limited accuracy and deficiency to uncover determinants of response. Here, we introduce an AI framework which addresses these issues.
Methods We present a new explainable deep learning framework based on transformer architecture1, 2 which combines data with different feature sets (including sparse date) and clinical endpoints for survival prediction or classification. This framework includes: (1) A new loss function based on a sigmoid approximation of Harrell’s concordance-index.3 (2) Explainability module providing feature importance and similarity score between features based on mutual contribution to predictions. (3) Transfer learning strategy to enable leveraging diverse clinical datasets in the public or private domain.
Results We utilized seven data sets comprising of more than 140,000 patients from IO, targeted and Chemotherapy treatments to benchmark our prediction models (table 1) in addition to 10 train/test splits performance evaluations.
Consistently, our framework outperformed other methods previously described in the literature, including CoxPH 4 and random survival forest.5, 6 For example, using the concordance index, our framework achieved 0.66 (0.04) vs. 0.60 (0.04) of the second-best method (Random survival forest in all cases) on MYSTIC IO arms clinical data. This improvement was a result of including transfer learning in the training process (table 2) which also achieved better performance in less training steps (figure 1).
Utilizing our Explainability module, we identified key features driving response prediction consistent with previous publications. For example, in Chowell et al. dataset, we identified Albumin, NLR, Chemo-before-IO-treatment and TMB as the most important features (figure 2). We also identified in sparse mutation calls of 469 genes from Samstein et al. dataset, functional modules of several genes only, each with a strong predictive power. For example, the functional module comprising of the genes: AKT2, BTK, CDC73, HLA-B, IKBKE, INPPL1, RFWD2, TRAF2 and WHSC1, related to adaptive immunity, stratified patients to two groups with a Hazard Ratio of 0.58 for the Samstein dataset and 0.42 for the validation dataset (figure 3).
Conclusions We propose a new framework with state-of-the-art performance in survival prediction and potential to uncover biological and clinical insights related to patient response and resistance. Importantly, our framework simplifies the process of translating complex AI models to clinical practice and may accelerate the benefit immunotherapy can bring to patients.
References
Vaswani Ashish, et al. Attention is all you need. Advances in Neural Information Processing Systems. 2017;30.
Devlin Jacob, et al. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
Schmid Matthias, Marvin N Wright, and Andreas Ziegler. On the use of Harrell’s C for clinical risk prediction via random survival forests. Expert Systems with Applications 2016;63:450–459.
Cox, David R. Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Methodological). 1972;34(2):187–202.
Ishwaran Hemant, et al. Random survival forests. The Annals of Applied Statistics. 2008;2(3):841-860.
Chowell Diego, et al. Improved prediction of immune checkpoint blockade efficacy across multiple cancer types. Nature Biotechnology. 2022;40(4):499–506.
Rizvi Naiyer A, et al. Durvalumab with or without tremelimumab vs standard chemotherapy in first-line treatment of metastatic non–small cell lung cancer: the MYSTIC phase 3 randomized clinical trial. JAMA oncology. 2020;6(5):661–674.
Rittmeyer Achim, et al. Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): a phase 3, open-label, multicentre randomised controlled trial. The Lancet. 2017;389(10066):255–265.
Samstein Robert M, et al. Tumor mutational load predicts survival after immunotherapy across multiple cancer types. Nature Genetics. 2019;51(2):202–206.
Thorsson Vésteinn, et al. The immune landscape of cancer. Immunity. 2018;48(4):812–830.
AACR Project Genie Consortium, et al. AACR Project GENIE: powering precision medicine through an international consortium. Cancer Discovery. 2017;7(8):818–831.
Miao Diana, et al. Genomic correlates of response to immune checkpoint blockade in microsatellite-stable solid tumors. Nature Genetics. 2018;50(9):1271–1281.